============================================================================== couchbase logs (indexer.log) cbbrowse_logs indexer.log ============================================================================== 2016-04-07T03:23:24.022-07:00 [Info] Indexer started with command line: [/opt/couchbase/bin/indexer -vbuckets=1024 -cluster=127.0.0.1:8091 -adminPort=9100 -scanPort=9101 -httpPort=9102 -streamInitPort=9103 -streamCatchupPort=9104 -streamMaintPort=9105 -storageDir=/data/@2i -diagDir=/opt/couchbase/var/lib/couchbase/crash] 2016-04-07T03:23:24.037-07:00 [Info] Indexer::NewIndexer Status INIT 2016-04-07T03:23:24.039-07:00 [Info] Setting maxcpus = 4 2016-04-07T03:23:24.039-07:00 [Info] Setting buffer block size to 16384 bytes 2016-04-07T03:23:24.039-07:00 [Info] Setting log level to Info 2016-04-07T03:23:24.039-07:00 [Info] Indexer::NewIndexer Starting with Vbuckets 1024 2016-04-07T03:23:24.062-07:00 [Info] [Queryport ":9101"] started ... 2016-04-07T03:23:24.063-07:00 [Info] New settings received: {"indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.log_level":"info","indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.compaction.min_frag":30,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.max_cpu_percent":400,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.memory_quota":31457280000} 2016-04-07T03:23:24.063-07:00 [Info] Setting maxcpus = 4 2016-04-07T03:23:24.063-07:00 [Info] Setting log level to Info 2016-04-07T03:23:24.832-07:00 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2016-04-07T03:23:24.848-07:00 [Info] ClustMgr:handleSetLocalValue Key IndexerId Value a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:24.848-07:00 [Info] Indexer Id a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:24.854-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:24.855-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:24.855-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.241:9999 2016-04-07T03:23:24.855-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:24.856-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:24.857-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:24.857-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.245:9999 2016-04-07T03:23:24.857-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:24.857-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:24.857-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err Post http://172.23.99.246:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.246:9999: connection refused 2016-04-07T03:23:24.857-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received Post http://172.23.99.246:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.246:9999: connection refused from 172.23.99.246:9999 2016-04-07T03:23:24.857-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:24.859-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:24.859-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.251:9999 2016-04-07T03:23:24.859-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:25.859-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received Post http://172.23.99.246:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.246:9999: connection refused 2016-04-07T03:23:25.859-07:00 [Error] Indexer::closeAllStreams Stream MAINT_STREAM Error from Projector Post http://172.23.99.246:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.246:9999: connection refused. Retrying. 2016-04-07T03:23:30.883-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:30.884-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:30.884-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.241:9999 2016-04-07T03:23:30.884-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:30.884-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:30.885-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:30.885-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.245:9999 2016-04-07T03:23:30.885-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:30.885-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:30.886-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:30.886-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.246:9999 2016-04-07T03:23:30.886-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:30.886-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:30.887-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:30.887-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.251:9999 2016-04-07T03:23:30.887-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:30.901-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.241:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:30.902-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.241:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:30.902-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.241:9999 2016-04-07T03:23:30.902-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:30.902-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.245:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:30.902-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.245:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:30.902-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.245:9999 2016-04-07T03:23:30.902-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:30.902-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.246:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:30.903-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.246:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:30.903-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.246:9999 2016-04-07T03:23:30.903-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:30.903-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.251:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:30.903-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.251:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:30.903-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.251:9999 2016-04-07T03:23:30.903-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:30.903-07:00 [Info] ClustMgr:handleGetGlobalTopology &{map[]} 2016-04-07T03:23:30.903-07:00 [Info] Indexer::initFromPersistedState Recovered IndexInstMap 2016-04-07T03:23:30.903-07:00 [Info] MutationMgr::handleUpdateIndexInstMap Message: MsgUpdateInstMap 2016-04-07T03:23:30.903-07:00 [Info] MutationMgr::handleUpdateIndexPartnMap Message: MsgUpdatePartnMap 2016-04-07T03:23:30.904-07:00 [Info] ClustMgr:handleIndexerReady &{52 [] {false false false false} 0} 2016-04-07T03:23:30.904-07:00 [Info] Indexer::NewIndexer Status ACTIVE 2016-04-07T03:23:30.904-07:00 [Info] CompactionManager: Refreshing settings 2016-04-07T03:23:34.072-07:00 [Info] New settings received: {"indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.log_level":"info","indexer.settings.compaction.min_frag":30,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.max_cpu_percent":400,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.memory_quota":31457280000} 2016-04-07T03:23:34.072-07:00 [Info] Setting maxcpus = 4 2016-04-07T03:23:34.072-07:00 [Info] Setting log level to Info 2016-04-07T03:23:34.072-07:00 [Info] CompactionManager: Refreshing settings 2016-04-07T03:23:34.09-07:00 [Info] New settings received: {"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_cpu_percent":2400,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":31457280000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":1000} 2016-04-07T03:23:34.09-07:00 [Info] Setting maxcpus = 24 2016-04-07T03:23:34.09-07:00 [Info] Setting log level to Info 2016-04-07T03:23:34.09-07:00 [Info] CompactionManager: Refreshing settings [goport] 2016/04/07 03:23:44 got new line on a stdin; terminating. 2016-04-07T03:23:50.651-07:00 [Info] Indexer started with command line: [/opt/couchbase/bin/indexer -vbuckets=1024 -cluster=127.0.0.1:8091 -adminPort=9100 -scanPort=9101 -httpPort=9102 -streamInitPort=9103 -streamCatchupPort=9104 -streamMaintPort=9105 -storageDir=/data/@2i -diagDir=/opt/couchbase/var/lib/couchbase/crash] 2016-04-07T03:23:50.665-07:00 [Info] Indexer::NewIndexer Status INIT 2016-04-07T03:23:50.672-07:00 [Info] Setting maxcpus = 24 2016-04-07T03:23:50.672-07:00 [Info] Setting buffer block size to 16384 bytes 2016-04-07T03:23:50.672-07:00 [Info] Setting log level to Info 2016-04-07T03:23:50.672-07:00 [Info] Indexer::NewIndexer Starting with Vbuckets 1024 2016-04-07T03:23:50.7-07:00 [Info] [Queryport ":9101"] started ... 2016-04-07T03:23:50.702-07:00 [Info] New settings received: {"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_cpu_percent":2400,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":31457280000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":1000} 2016-04-07T03:23:50.702-07:00 [Info] Setting maxcpus = 24 2016-04-07T03:23:50.702-07:00 [Info] Setting log level to Info 2016-04-07T03:23:51.479-07:00 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2016-04-07T03:23:51.488-07:00 [Info] Indexer Id a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:51.501-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:51.501-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err Post http://172.23.99.241:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.241:9999: connection refused 2016-04-07T03:23:51.501-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received Post http://172.23.99.241:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.241:9999: connection refused from 172.23.99.241:9999 2016-04-07T03:23:51.501-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:51.502-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err Post http://172.23.99.245:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.245:9999: connection refused 2016-04-07T03:23:51.502-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received Post http://172.23.99.245:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.245:9999: connection refused from 172.23.99.245:9999 2016-04-07T03:23:51.502-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:51.502-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err Post http://172.23.99.246:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.246:9999: connection refused 2016-04-07T03:23:51.502-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received Post http://172.23.99.246:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.246:9999: connection refused from 172.23.99.246:9999 2016-04-07T03:23:51.502-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:51.502-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err Post http://172.23.99.251:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.251:9999: connection refused 2016-04-07T03:23:51.502-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received Post http://172.23.99.251:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.251:9999: connection refused from 172.23.99.251:9999 2016-04-07T03:23:52.502-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received Post http://172.23.99.251:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.251:9999: connection refused 2016-04-07T03:23:52.503-07:00 [Error] Indexer::closeAllStreams Stream MAINT_STREAM Error from Projector Post http://172.23.99.251:9999/adminport/shutdownTopicRequest: dial tcp 172.23.99.251:9999: connection refused. Retrying. 2016-04-07T03:23:57.517-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:57.519-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:57.519-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.241:9999 2016-04-07T03:23:57.519-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:57.519-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:57.52-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:57.52-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.245:9999 2016-04-07T03:23:57.52-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:57.52-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:57.522-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:57.522-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.246:9999 2016-04-07T03:23:57.522-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:57.522-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:57.523-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:57.523-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.251:9999 2016-04-07T03:23:57.523-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:57.535-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.241:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:57.535-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.241:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:57.535-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.241:9999 2016-04-07T03:23:57.535-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:57.535-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.245:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:57.536-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.245:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:57.536-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.245:9999 2016-04-07T03:23:57.536-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:57.536-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.246:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:57.536-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.246:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:57.536-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.246:9999 2016-04-07T03:23:57.536-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:57.536-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.251:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:23:57.537-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.251:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:23:57.537-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.251:9999 2016-04-07T03:23:57.537-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:23:57.537-07:00 [Info] ClustMgr:handleGetGlobalTopology &{map[]} 2016-04-07T03:23:57.537-07:00 [Info] Indexer::initFromPersistedState Recovered IndexInstMap 2016-04-07T03:23:57.537-07:00 [Info] MutationMgr::handleUpdateIndexInstMap Message: MsgUpdateInstMap 2016-04-07T03:23:57.537-07:00 [Info] MutationMgr::handleUpdateIndexPartnMap Message: MsgUpdatePartnMap 2016-04-07T03:23:57.537-07:00 [Info] ClustMgr:handleIndexerReady &{52 [] {false false false false} 0} 2016-04-07T03:23:57.537-07:00 [Info] Indexer::NewIndexer Status ACTIVE 2016-04-07T03:23:57.537-07:00 [Info] CompactionManager: Refreshing settings 2016-04-07T03:24:51.48-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":143360,"needs_restart":false,"num_connections":0,"timings/stats_response":"55 2246503 120385513101"} 2016-04-07T03:25:51.48-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":143360,"needs_restart":false,"num_connections":0,"timings/stats_response":"127 4577829 209697280403"} [goport] 2016/04/07 03:25:55 got new line on a stdin; terminating. 2016-04-07T03:26:05.009-07:00 [Info] Indexer started with command line: [/opt/couchbase/bin/indexer -vbuckets=1024 -cluster=127.0.0.1:8091 -adminPort=9100 -scanPort=9101 -httpPort=9102 -streamInitPort=9103 -streamCatchupPort=9104 -streamMaintPort=9105 -storageDir=/data/@2i -diagDir=/opt/couchbase/var/lib/couchbase/crash] 2016-04-07T03:26:05.026-07:00 [Info] Indexer::NewIndexer Status INIT 2016-04-07T03:26:05.032-07:00 [Info] Setting maxcpus = 24 2016-04-07T03:26:05.032-07:00 [Info] Setting buffer block size to 16384 bytes 2016-04-07T03:26:05.032-07:00 [Info] Setting log level to Info 2016-04-07T03:26:05.032-07:00 [Info] Indexer::NewIndexer Starting with Vbuckets 1024 2016-04-07T03:26:05.06-07:00 [Info] [Queryport ":9101"] started ... 2016-04-07T03:26:05.061-07:00 [Info] New settings received: {"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_cpu_percent":2400,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":31457280000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":1000} 2016-04-07T03:26:05.062-07:00 [Info] Setting maxcpus = 24 2016-04-07T03:26:05.062-07:00 [Info] Setting log level to Info 2016-04-07T03:26:05.78-07:00 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2016-04-07T03:26:05.781-07:00 [Info] Indexer Id a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:05.796-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:05.798-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:05.798-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.241:9999 2016-04-07T03:26:05.799-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:05.799-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:05.8-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:05.8-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.245:9999 2016-04-07T03:26:05.8-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:05.8-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:05.802-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:05.802-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.246:9999 2016-04-07T03:26:05.802-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:05.802-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:05.803-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:05.803-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.251:9999 2016-04-07T03:26:05.803-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:05.817-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.241:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:05.817-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.241:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:05.817-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.241:9999 2016-04-07T03:26:05.817-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:05.817-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.245:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:05.818-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.245:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:05.818-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.245:9999 2016-04-07T03:26:05.818-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:05.818-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.246:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:05.818-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.246:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:05.818-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.246:9999 2016-04-07T03:26:05.818-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:05.818-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.251:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:05.819-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.251:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:05.819-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.251:9999 2016-04-07T03:26:05.819-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:05.819-07:00 [Info] ClustMgr:handleGetGlobalTopology &{map[]} 2016-04-07T03:26:05.819-07:00 [Info] Indexer::initFromPersistedState Recovered IndexInstMap 2016-04-07T03:26:05.819-07:00 [Info] MutationMgr::handleUpdateIndexInstMap Message: MsgUpdateInstMap 2016-04-07T03:26:05.819-07:00 [Info] MutationMgr::handleUpdateIndexPartnMap Message: MsgUpdatePartnMap 2016-04-07T03:26:05.819-07:00 [Info] ClustMgr:handleIndexerReady &{52 [] {false false false false} 0} 2016-04-07T03:26:05.819-07:00 [Info] Indexer::NewIndexer Status ACTIVE 2016-04-07T03:26:05.819-07:00 [Info] CompactionManager: Refreshing settings [goport] 2016/04/07 03:26:12 got new line on a stdin; terminating. 2016-04-07T03:26:12.127-07:00 [Info] Indexer started with command line: [/opt/couchbase/bin/indexer -vbuckets=1024 -cluster=127.0.0.1:8091 -adminPort=9100 -scanPort=9101 -httpPort=9102 -streamInitPort=9103 -streamCatchupPort=9104 -streamMaintPort=9105 -storageDir=/data/@2i -diagDir=/opt/couchbase/var/lib/couchbase/crash] 2016-04-07T03:26:12.14-07:00 [Info] Indexer::NewIndexer Status INIT 2016-04-07T03:26:12.142-07:00 [Info] Setting maxcpus = 24 2016-04-07T03:26:12.142-07:00 [Info] Setting buffer block size to 16384 bytes 2016-04-07T03:26:12.142-07:00 [Info] Setting log level to Info 2016-04-07T03:26:12.142-07:00 [Info] Indexer::NewIndexer Starting with Vbuckets 1024 2016-04-07T03:26:12.165-07:00 [Info] [Queryport ":9101"] started ... 2016-04-07T03:26:12.166-07:00 [Info] New settings received: {"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_cpu_percent":2400,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":31457280000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":1000} 2016-04-07T03:26:12.167-07:00 [Info] Setting maxcpus = 24 2016-04-07T03:26:12.167-07:00 [Info] Setting log level to Info 2016-04-07T03:26:12.868-07:00 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2016-04-07T03:26:12.868-07:00 [Info] Indexer Id a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:12.883-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:12.885-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:12.885-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.241:9999 2016-04-07T03:26:12.885-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:12.885-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:12.886-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:12.886-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.245:9999 2016-04-07T03:26:12.886-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:12.887-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:12.888-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:12.888-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.246:9999 2016-04-07T03:26:12.888-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:12.888-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:12.889-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:12.889-07:00 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.23.99.251:9999 2016-04-07T03:26:12.889-07:00 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:12.904-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.241:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:12.904-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.241:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:12.904-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.241:9999 2016-04-07T03:26:12.904-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:12.904-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.245:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:12.905-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.245:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:12.905-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.245:9999 2016-04-07T03:26:12.905-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:12.905-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.246:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:12.905-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.246:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:12.905-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.246:9999 2016-04-07T03:26:12.905-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:12.905-07:00 [Info] KVSender::sendShutdownTopic Projector 172.23.99.251:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae 2016-04-07T03:26:12.906-07:00 [Fatal] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.23.99.251:9999 Topic INIT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae. Err projector.topicMissing 2016-04-07T03:26:12.906-07:00 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.23.99.251:9999 2016-04-07T03:26:12.906-07:00 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2016-04-07T03:26:12.906-07:00 [Info] ClustMgr:handleGetGlobalTopology &{map[]} 2016-04-07T03:26:12.906-07:00 [Info] Indexer::initFromPersistedState Recovered IndexInstMap 2016-04-07T03:26:12.906-07:00 [Info] MutationMgr::handleUpdateIndexInstMap Message: MsgUpdateInstMap 2016-04-07T03:26:12.906-07:00 [Info] MutationMgr::handleUpdateIndexPartnMap Message: MsgUpdatePartnMap 2016-04-07T03:26:12.906-07:00 [Info] ClustMgr:handleIndexerReady &{52 [] {false false false false} 0} 2016-04-07T03:26:12.906-07:00 [Info] Indexer::NewIndexer Status ACTIVE 2016-04-07T03:26:12.906-07:00 [Info] CompactionManager: Refreshing settings 2016-04-07T03:26:42.009-07:00 [Error] invalid config param "indexer.settings.gc_percent" 2016-04-07T03:26:42.018-07:00 [Error] invalid config param "indexer.settings.storage_mode" 2016-04-07T03:27:12.868-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"72 2587590 125659731758"} 2016-04-07T03:28:12.883-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"143 4539687 186325192557"} 2016-04-07T03:29:12.883-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"216 6642973 259132643619"} 2016-04-07T03:30:12.893-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"288 9133504 361113942522"} 2016-04-07T03:31:12.893-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"360 11571968 456890311372"} 2016-04-07T03:32:12.907-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"432 13981235 554965308739"} 2016-04-07T03:33:12.907-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"504 16513100 661254435874"} 2016-04-07T03:34:12.923-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"576 19176814 778688860028"} 2016-04-07T03:35:12.923-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"648 21585730 877768414220"} 2016-04-07T03:36:12.931-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"720 24523220 1018219180796"} 2016-04-07T03:37:12.931-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"792 27001688 1116371258672"} 2016-04-07T03:38:12.932-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"864 29576677 1228802229735"} 2016-04-07T03:39:12.939-07:00 [Info] PeriodicStats = {"memory_quota":31457280000,"memory_used":192512,"needs_restart":false,"num_connections":0,"timings/stats_response":"935 32101711 1333427398533"} 2016-04-07T03:39:19.442-07:00 [Info] clustMgrAgent::OnIndexCreate Notification Received for Create Index DefnId: 9366861204008862690 Name: myindex Using: forestdb Bucket: bucket-1 IsPrimary: false SecExprs: [`alt_email`] PartitionScheme: SINGLE PartitionKey: partn WhereExpr: 2016-04-07T03:39:19.443-07:00 [Info] Indexer::handleCreateIndex InstId: 9366861204008862690 Defn: DefnId: 9366861204008862690 Name: myindex Using: forestdb Bucket: bucket-1 IsPrimary: false SecExprs: [`alt_email`] PartitionScheme: SINGLE PartitionKey: partn WhereExpr: State: INDEX_STATE_CREATED Stream: NIL_STREAM PartitionContainer: &{map[1:{1 [:9105]}] 1} 2016-04-07T03:39:19.457-07:00 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9366861204008862690 Partition: PartitionId: 1 Endpoints: [:9105] 2016-04-07T03:39:19.463-07:00 [Info] ForestDBSlice:NewForestDBSlice Created New Slice Id 0 IndexInstId 9366861204008862690 WriterThreads 1 2016-04-07T03:39:19.463-07:00 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9366861204008862690 Slice: SliceId: 0 File: /data/@2i/bucket-1_myindex_9366861204008862690_0.index Index: 9366861204008862690 2016-04-07T03:39:19.463-07:00 [Info] MutationMgr::handleUpdateIndexInstMap Message: MsgUpdateInstMap InstanceId: 9366861204008862690 Name: myindex Bucket: bucket-1 State: INDEX_STATE_CREATED Stream: NIL_STREAM 2016-04-07T03:39:19.463-07:00 [Info] MutationMgr::handleUpdateIndexPartnMap Message: MsgUpdatePartnMap InstanceId: 9366861204008862690 PartitionId: 0 Endpoints: [:9105] 2016-04-07T03:39:19.463-07:00 [Info] clustMgrAgent::OnIndexCreate Success for Create Index DefnId: 9366861204008862690 Name: myindex Using: forestdb Bucket: bucket-1 IsPrimary: false SecExprs: [`alt_email`] PartitionScheme: SINGLE PartitionKey: partn WhereExpr: 2016-04-07T03:39:19.466-07:00 [Info] messageListener.start(): message channel closed. Remove peer 172.23.99.241/22:indexer:MetadataProvider:c4:26:5d:44:a5:ff:9b:73 as follower. 2016-04-07T03:39:23.088-07:00 [Info] DCPT[secidx:getseqnos-c:90:69:f1:1c:7a:f3:52] ##abba feed started ... 2016-04-07T03:39:23.091-07:00 [Info] DCPT[secidx:getseqnos-4e:8f:9f:7d:57:42:1b:17] ##abba feed started ... 2016-04-07T03:39:23.093-07:00 [Info] DCPT[secidx:getseqnos-94:38:63:88:1f:ff:b0:c7] ##abba feed started ... 2016-04-07T03:39:23.095-07:00 [Info] DCPT[secidx:getseqnos-22:73:d8:6:e4:97:b1:52] ##abba feed started ... 2016-04-07T03:39:23.095-07:00 [Info] {bucket,feeds} "bucket-1" created for dcp_seqno cache... 2016-04-07T03:39:29.565-07:00 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [9366861204008862690] 2016-04-07T03:39:29.565-07:00 [Info] Indexer::handleBuildIndex [9366861204008862690] 2016-04-07T03:39:29.584-07:00 [Info] Indexer::handleBuildIndex Added Index: [9366861204008862690] to Stream: MAINT_STREAM State: INDEX_STATE_INITIAL 2016-04-07T03:39:29.584-07:00 [Info] MutationMgr::handleUpdateIndexInstMap Message: MsgUpdateInstMap InstanceId: 9366861204008862690 Name: myindex Bucket: bucket-1 State: INDEX_STATE_INITIAL Stream: MAINT_STREAM 2016-04-07T03:39:29.585-07:00 [Info] Timekeeper::handleStreamOpen Stream MAINT_STREAM State Changed to ACTIVE 2016-04-07T03:39:29.585-07:00 [Info] StreamState::initBucketInStream New Bucket bucket-1 Added for Stream MAINT_STREAM 2016-04-07T03:39:29.585-07:00 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket-1:1]] 2016-04-07T03:39:29.585-07:00 [Info] Timekeeper::addIndextoStream add BuildInfo index 9366861204008862690 stream MAINT_STREAM bucket bucket-1 state INDEX_STATE_INITIAL 2016-04-07T03:39:29.585-07:00 [Info] Timekeeper::startTimer MAINT_STREAM bucket-1 2016-04-07T03:39:29.585-07:00 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM Bucket: bucket-1 BuildTS: [0=19535,1=19525,2=19535,3=19530,4=19535,5=19530,6=19535,7=19525,8=19535,9=19525,10=19535,11=19530,12=19535,13=19530,14=19535,15=19525,16=19535,17=19525,18=19536,19=19530,20=19535,21=19530,22=19535,23=19525,24=19535,25=19525,26=19535,27=19530,28=19535,29=19530,30=19535,31=19525,32=19535,33=19525,34=19535,35=19530,36=19535,37=19530,38=19535,39=19525,40=19535,41=19525,42=19535,43=19530,44=19535,45=19530,46=19535,47=19525,48=19535,49=19525,50=19535,51=19530,52=19535,53=19530,54=19535,55=19525,56=19535,57=19525,58=19535,59=19530,60=19535,61=19530,62=19535,63=19525,64=19530,65=19535,66=19525,67=19535,68=19525,69=19535,70=19530,71=19535,72=19530,73=19535,74=19525,75=19535,76=19525,77=19535,78=19530,79=19535,80=19530,81=19535,82=19525,83=19535,84=19525,85=19535,86=19530,87=19535,88=19530,89=19535,90=19525,91=19535,92=19525,93=19535,94=19530,95=19535,96=19530,97=19535,98=19525,99=19535,100=19525,101=19535,102=19530,103=19535,104=19530,105=19535,106=19525,107=19535,108=19525,109=19535,110=19530,111=19535,112=19530,113=19535,114=19525,115=19535,116=19525,117=19535,118=19530,119=19535,120=19530,121=19535,122=19525,123=19535,124=19525,125=19535,126=19530,127=19535,128=19535,129=19530,130=19535,131=19525,132=19535,133=19525,134=19535,135=19530,136=19535,137=19530,138=19535,139=19525,140=19535,141=19525,142=19535,143=19530,144=19535,145=19530,146=19535,147=19525,148=19535,149=19525,150=19535,151=19530,152=19535,153=19530,154=19535,155=19525,156=19535,157=19525,158=19535,159=19530,160=19535,161=19530,162=19535,163=19525,164=19535,165=19525,166=19535,167=19530,168=19535,169=19530,170=19535,171=19525,172=19535,173=19525,174=19535,175=19530,176=19535,177=19530,178=19535,179=19525,180=19535,181=19525,182=19535,183=19530,184=19535,185=19530,186=19535,187=19525,188=19535,189=19525,190=19535,191=19530,192=19525,193=19535,194=19530,195=19535,196=19530,197=19535,198=19525,199=19535,200=19525,201=19535,202=19530,203=19535,204=19530,205=19535,206=19525,207=19535,208=19525,209=19535,210=19530,211=19535,212=19530,213=19535,214=19525,215=19535,216=19525,217=19535,218=19530,219=19535,220=19530,221=19535,222=19525,223=19535,224=19525,225=19535,226=19530,227=19535,228=19530,229=19535,230=19525,231=19535,232=19525,233=19535,234=19530,235=19535,236=19530,237=19535,238=19525,239=19535,240=19525,241=19535,242=19530,243=19535,244=19530,245=19535,246=19525,247=19535,248=19525,249=19535,250=19530,251=19535,252=19530,253=19535,254=19525,255=19535,256=19530,257=19535,258=19525,259=19535,260=19525,261=19535,262=19530,263=19535,264=19530,265=19535,266=19525,267=19535,268=19525,269=19535,270=19530,271=19535,272=19530,273=19535,274=19525,275=19535,276=19525,277=19535,278=19530,279=19535,280=19530,281=19535,282=19525,283=19535,284=19525,285=19535,286=19530,287=19535,288=19530,289=19535,290=19525,291=19535,292=19525,293=19535,294=19530,295=19535,296=19530,297=19535,298=19525,299=19535,300=19525,301=19535,302=19530,303=19535,304=19530,305=19535,306=19525,307=19535,308=19525,309=19535,310=19530,311=19535,312=19530,313=19535,314=19525,315=19535,316=19525,317=19535,318=19530,319=19535,320=19535,321=19525,322=19535,323=19530,324=19535,325=19530,326=19535,327=19525,328=19535,329=19525,330=19535,331=19530,332=19535,333=19530,334=19535,335=19525,336=19535,337=19525,338=19535,339=19530,340=19535,341=19530,342=19535,343=19525,344=19535,345=19525,346=19535,347=19530,348=19535,349=19530,350=19535,351=19525,352=19535,353=19525,354=19535,355=19530,356=19535,357=19530,358=19535,359=19525,360=19535,361=19525,362=19535,363=19530,364=19535,365=19530,366=19535,367=19525,368=19535,369=19525,370=19535,371=19530,372=19535,373=19530,374=19535,375=19525,376=19535,377=19525,378=19535,379=19530,380=19535,381=19530,382=19535,383=19525,384=19525,385=19535,386=19530,387=19535,388=19530,389=19535,390=19525,391=19535,392=19525,393=19535,394=19530,395=19535,396=19530,397=19535,398=19525,399=19535,400=19525,401=19535,402=19530,403=19535,404=19530,405=19535,406=19525,407=19535,408=19525,409=19535,410=19530,411=19535,412=19530,413=19535,414=19525,415=19535,416=19525,417=19535,418=19530,419=19535,420=19530,421=19535,422=19525,423=19535,424=19525,425=19535,426=19530,427=19535,428=19530,429=19535,430=19525,431=19535,432=19525,433=19535,434=19530,435=19535,436=19530,437=19535,438=19525,439=19535,440=19525,441=19535,442=19530,443=19535,444=19530,445=19535,446=19525,447=19535,448=19535,449=19530,450=19535,451=19525,452=19535,453=19525,454=19535,455=19530,456=19535,457=19530,458=19535,459=19525,460=19535,461=19525,462=19535,463=19530,464=19535,465=19530,466=19535,467=19525,468=19535,469=19525,470=19535,471=19530,472=19535,473=19530,474=19535,475=19525,476=19535,477=19525,478=19535,479=19530,480=19535,481=19530,482=19535,483=19525,484=19535,485=19525,486=19535,487=19530,488=19535,489=19530,490=19535,491=19525,492=19535,493=19525,494=19535,495=19530,496=19535,497=19530,498=19535,499=19525,500=19535,501=19525,502=19535,503=19530,504=19535,505=19530,506=19535,507=19525,508=19535,509=19525,510=19535,511=19530,512=19535,513=19530,514=19535,515=19525,516=19535,517=19525,518=19535,519=19530,520=19535,521=19530,522=19535,523=19525,524=19535,525=19525,526=19535,527=19530,528=19535,529=19530,530=19535,531=19525,532=19535,533=19525,534=19535,535=19530,536=19535,537=19530,538=19535,539=19525,540=19535,541=19525,542=19535,543=19530,544=19535,545=19530,546=19535,547=19525,548=19535,549=19525,550=19535,551=19530,552=19535,553=19530,554=19535,555=19525,556=19535,557=19525,558=19535,559=19530,560=19535,561=19530,562=19535,563=19525,564=19535,565=19525,566=19535,567=19529,568=19535,569=19530,570=19535,571=19525,572=19535,573=19525,574=19535,575=19530,576=19525,577=19535,578=19530,579=19535,580=19530,581=19535,582=19525,583=19535,584=19525,585=19535,586=19530,587=19535,588=19530,589=19535,590=19525,591=19535,592=19525,593=19535,594=19530,595=19535,596=19530,597=19535,598=19525,599=19535,600=19525,601=19535,602=19530,603=19535,604=19530,605=19535,606=19525,607=19535,608=19525,609=19535,610=19530,611=19535,612=19530,613=19535,614=19525,615=19535,616=19525,617=19535,618=19530,619=19535,620=19530,621=19535,622=19525,623=19535,624=19525,625=19535,626=19530,627=19535,628=19530,629=19535,630=19525,631=19535,632=19525,633=19535,634=19530,635=19535,636=19530,637=19535,638=19525,639=19535,640=19535,641=19525,642=19535,643=19530,644=19535,645=19530,646=19535,647=19525,648=19535,649=19525,650=19535,651=19530,652=19535,653=19530,654=19535,655=19525,656=19535,657=19525,658=19535,659=19530,660=19535,661=19530,662=19535,663=19525,664=19535,665=19525,666=19535,667=19530,668=19535,669=19530,670=19535,671=19525,672=19535,673=19525,674=19535,675=19530,676=19535,677=19530,678=19535,679=19525,680=19535,681=19525,682=19535,683=19530,684=19535,685=19530,686=19535,687=19525,688=19535,689=19525,690=19535,691=19530,692=19535,693=19530,694=19535,695=19525,696=19535,697=19525,698=19535,699=19530,700=19535,701=19530,702=19535,703=19525,704=19530,705=19535,706=19525,707=19535,708=19525,709=19535,710=19530,711=19535,712=19530,713=19535,714=19525,715=19535,716=19525,717=19535,718=19530,719=19535,720=19530,721=19535,722=19525,723=19535,724=19525,725=19535,726=19530,727=19535,728=19530,729=19535,730=19525,731=19535,732=19525,733=19535,734=19530,735=19535,736=19530,737=19535,738=19525,739=19535,740=19525,741=19535,742=19530,743=19535,744=19530,745=19535,746=19525,747=19535,748=19525,749=19535,750=19530,751=19535,752=19530,753=19535,754=19525,755=19535,756=19525,757=19535,758=19530,759=19535,760=19530,761=19535,762=19525,763=19535,764=19525,765=19535,766=19530,767=19535,768=19525,769=19535,770=19530,771=19535,772=19530,773=19535,774=19525,775=19535,776=19525,777=19535,778=19530,779=19535,780=19530,781=19535,782=19525,783=19535,784=19525,785=19535,786=19530,787=19535,788=19530,789=19535,790=19525,791=19535,792=19525,793=19535,794=19530,795=19535,796=19530,797=19535,798=19525,799=19535,800=19525,801=19535,802=19530,803=19535,804=19530,805=19535,806=19525,807=19535,808=19525,809=19535,810=19530,811=19535,812=19530,813=19535,814=19525,815=19535,816=19525,817=19535,818=19530,819=19535,820=19530,821=19535,822=19525,823=19535,824=19525,825=19535,826=19530,827=19535,828=19530,8 29=19535,830=19525,831=19535,832=19535,833=19530,834=19535,835=19525,836=19535,837=19525,838=19535,839=19530,840=19535,841=19530,842=19535,843=19525,844=19535,845=19525,846=19535,847=19530,848=19535,849=19530,850=19535,851=19525,852=19535,853=19525,854=19535,855=19530,856=19535,857=19530,858=19535,859=19525,860=19535,861=19525,862=19535,863=19530,864=19535,865=19530,866=19535,867=19525,868=19535,869=19525,870=19535,871=19530,872=19535,873=19530,874=19535,875=19525,876=19535,877=19525,878=19535,879=19530,880=19535,881=19530,882=19535,883=19525,884=19535,885=19525,886=19535,887=19530,888=19535,889=19530,890=19535,891=19525,892=19535,893=19525,894=19535,895=19530,896=19530,897=19535,898=19525,899=19535,900=19525,901=19535,902=19530,903=19535,904=19530,905=19535,906=19525,907=19535,908=19525,909=19535,910=19530,911=19535,912=19530,913=19535,914=19525,915=19535,916=19525,917=19535,918=19530,919=19535,920=19530,921=19535,922=19525,923=19535,924=19525,925=19535,926=19530,927=19535,928=19530,929=19535,930=19525,931=19535,932=19525,933=19535,934=19530,935=19535,936=19530,937=19535,938=19525,939=19535,940=19525,941=19535,942=19530,943=19535,944=19530,945=19535,946=19525,947=19535,948=19525,949=19535,950=19530,951=19535,952=19530,953=19535,954=19525,955=19535,956=19525,957=19535,958=19530,959=19535,960=19535,961=19525,962=19535,963=19530,964=19535,965=19530,966=19535,967=19525,968=19535,969=19525,970=19535,971=19530,972=19535,973=19530,974=19535,975=19525,976=19535,977=19525,978=19535,979=19530,980=19535,981=19530,982=19535,983=19525,984=19535,985=19525,986=19535,987=19530,988=19535,989=19530,990=19535,991=19525,992=19535,993=19525,994=19535,995=19530,996=19535,997=19530,998=19535,999=19525,1000=19535,1001=19525,1002=19535,1003=19530,1004=19535,1005=19530,1006=19535,1007=19525,1008=19535,1009=19525,1010=19535,1011=19530,1012=19535,1013=19530,1014=19535,1015=19525,1016=19535,1017=19525,1018=19535,1019=19530,1020=19535,1021=19530,1022=19535,1023=19525] IndexList: [ InstId: 9366861204008862690 Defn: DefnId: 9366861204008862690 Name: myindex Using: forestdb Bucket: bucket-1 IsPrimary: false SecExprs: [`alt_email`] PartitionScheme: SINGLE PartitionKey: partn WhereExpr: State: INDEX_STATE_INITIAL Stream: MAINT_STREAM PartitionContainer: &{map[1:{1 [:9105]}] 1}] RestartTs: 2016-04-07T03:39:29.592-07:00 [Info] MutationMgr:: Set maxVbQueueLength 10000 2016-04-07T03:39:29.593-07:00 [Info] DATP[->dataport ":9105"] started ... 2016-04-07T03:39:29.593-07:00 [Info] MutationStreamReader::startWorkers Starting All Stream Workers MAINT_STREAM 2016-04-07T03:39:29.593-07:00 [Info] MutationStreamReader::startMutationStreamWorker Stream Worker 0 Started for Stream MAINT_STREAM. 2016-04-07T03:39:29.593-07:00 [Info] ClustMgr:handleUpdateTopologyForIndex &{47 [{9366861204008862690 {9366861204008862690 myindex forestdb bucket-1 8bc620dc3c466302736c51871ada764f false [`alt_email`] N1QL SINGLE partn true [a0:85:3b:34:1d:f9:92:ae]} 2 1 0xc20bfb35e0 []}] {true true false true} 0} 2016-04-07T03:39:29.593-07:00 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [9366861204008862690] 2016-04-07T03:39:29.596-07:00 [Info] messageListener.start(): message channel closed. Remove peer 172.23.99.241/22:indexer:MetadataProvider:43:71:c2:7f:4f:b5:d3:62 as follower. 2016-04-07T03:39:29.891-07:00 [Info] KVSender::sendMutationTopicRequest Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae bucket-1 Instances [indexInstance: singlePartn: > ] 2016-04-07T03:39:29.901-07:00 [Info] DATP[->dataport ":9105"] new connection "172.23.99.241:54143" +1 2016-04-07T03:39:30.111-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1 250101287947860 0 2016-04-07T03:39:30.111-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1 2016-04-07T03:39:30.111-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 8 207417235595714 0 2016-04-07T03:39:30.111-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 4 30074042751170 0 2016-04-07T03:39:30.111-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 5 106072919270093 0 2016-04-07T03:39:30.112-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 6 128481424948410 0 2016-04-07T03:39:30.112-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 7 241865061742433 0 2016-04-07T03:39:30.112-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 9 201158734268790 0 2016-04-07T03:39:30.112-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 0 133008252273043 0 2016-04-07T03:39:30.112-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 2 137519925955180 0 2016-04-07T03:39:30.112-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 3 73744122958337 0 2016-04-07T03:39:30.112-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 10 83350529381168 0 2016-04-07T03:39:30.136-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 11 49328356041427 0 2016-04-07T03:39:30.136-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 12 102540637289912 0 2016-04-07T03:39:30.136-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 13 142866659728517 0 2016-04-07T03:39:30.161-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 14 192528801138597 0 2016-04-07T03:39:30.161-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 15 218402253690163 0 2016-04-07T03:39:30.183-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 16 32070096674356 0 2016-04-07T03:39:30.183-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 17 227000729252625 0 2016-04-07T03:39:30.206-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1 QueuedCount 8 2016-04-07T03:39:30.207-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:30.211-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 18 81176371536798 0 2016-04-07T03:39:30.211-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 19 93237485333105 0 2016-04-07T03:39:30.236-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 20 174259526334228 0 2016-04-07T03:39:30.258-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 21 95609095456021 0 2016-04-07T03:39:30.258-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 22 217608141690837 0 2016-04-07T03:39:30.258-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 23 117896686640076 0 2016-04-07T03:39:30.26-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 24 251076590802331 0 2016-04-07T03:39:30.275-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 28 74392847687650 0 2016-04-07T03:39:30.276-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 25 28961404527767 0 2016-04-07T03:39:30.276-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 26 30276927640269 0 2016-04-07T03:39:30.276-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 27 148675715586920 0 2016-04-07T03:39:30.287-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 29 23022960312890 0 2016-04-07T03:39:30.287-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 30 225516218512617 0 2016-04-07T03:39:30.298-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 33 66513892927262 0 2016-04-07T03:39:30.298-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 34 155074203457101 0 2016-04-07T03:39:30.299-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 31 212329657445657 0 2016-04-07T03:39:30.299-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 32 96203910132138 0 2016-04-07T03:39:30.31-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 35 65108598077089 0 2016-04-07T03:39:30.31-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 36 151648285816513 0 2016-04-07T03:39:30.31-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 37 78269818083246 0 2016-04-07T03:39:30.324-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 38 150285349026004 0 2016-04-07T03:39:30.324-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 39 50123939355425 0 2016-04-07T03:39:30.325-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 40 148455974406062 0 2016-04-07T03:39:30.337-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 41 91171252419328 0 2016-04-07T03:39:30.337-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 42 215441880526062 0 2016-04-07T03:39:30.337-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 43 160237491809349 0 2016-04-07T03:39:30.348-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 44 64062603498698 0 2016-04-07T03:39:30.348-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 45 141248109243543 0 2016-04-07T03:39:30.348-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 46 239222454672961 0 2016-04-07T03:39:30.348-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 47 159187251750693 0 2016-04-07T03:39:30.36-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 49 126284296966452 0 2016-04-07T03:39:30.36-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 50 223839724642926 0 2016-04-07T03:39:30.36-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 51 170016639035719 0 2016-04-07T03:39:30.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 48 173732365344444 0 2016-04-07T03:39:30.372-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 52 94108570624106 0 2016-04-07T03:39:30.372-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 54 9852741336729 0 2016-04-07T03:39:30.372-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 55 245578824341481 0 2016-04-07T03:39:30.373-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 53 33296395917523 0 2016-04-07T03:39:30.383-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 58 221358905098755 0 2016-04-07T03:39:30.383-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 57 176721550997745 0 2016-04-07T03:39:30.383-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 59 128458042429656 0 2016-04-07T03:39:30.384-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 56 219349427216254 0 2016-04-07T03:39:30.386-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 60 262946772097074 0 2016-04-07T03:39:30.396-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 63 24941884172272 0 2016-04-07T03:39:30.396-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 64 191332750399812 0 2016-04-07T03:39:30.397-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 61 206606184019272 0 2016-04-07T03:39:30.397-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 62 213450210130952 0 2016-04-07T03:39:30.406-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 67 148556568496620 0 2016-04-07T03:39:30.406-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 65 81297193235912 0 2016-04-07T03:39:30.406-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 66 195904515584123 0 2016-04-07T03:39:30.41-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:30.422-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 68 155055448248896 0 2016-04-07T03:39:30.423-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 69 53442312770458 0 2016-04-07T03:39:30.423-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 70 37508501045023 0 2016-04-07T03:39:30.423-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 71 103791648028293 0 2016-04-07T03:39:30.433-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 72 129590897115147 0 2016-04-07T03:39:30.433-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 73 90876146520970 0 2016-04-07T03:39:30.433-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 74 90711732721271 0 2016-04-07T03:39:30.446-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 77 249673248992849 0 2016-04-07T03:39:30.446-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 75 8869606298004 0 2016-04-07T03:39:30.446-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 76 177394035940317 0 2016-04-07T03:39:30.459-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 78 11523502974855 0 2016-04-07T03:39:30.459-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 79 235225734911825 0 2016-04-07T03:39:30.459-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 80 222642341066683 0 2016-04-07T03:39:30.475-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 81 34991344654551 0 2016-04-07T03:39:30.475-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 82 206288673760689 0 2016-04-07T03:39:30.475-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 83 163881224551297 0 2016-04-07T03:39:30.485-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 85 56658965101147 0 2016-04-07T03:39:30.485-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 86 96029505058755 0 2016-04-07T03:39:30.485-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 84 266428335947468 0 2016-04-07T03:39:30.485-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 87 200658156679773 0 2016-04-07T03:39:30.493-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 91 109496686622288 0 2016-04-07T03:39:30.494-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 88 60404247365987 0 2016-04-07T03:39:30.494-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 89 273274919463019 0 2016-04-07T03:39:30.494-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 90 107957390414064 0 2016-04-07T03:39:30.501-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 92 109073615124127 0 2016-04-07T03:39:30.501-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 93 84288282955981 0 2016-04-07T03:39:30.511-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 94 204935329039901 0 2016-04-07T03:39:30.511-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 95 113885358193829 0 2016-04-07T03:39:30.511-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 96 232869322980416 0 2016-04-07T03:39:30.522-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 97 152923237115309 0 2016-04-07T03:39:30.522-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 98 209520741359258 0 2016-04-07T03:39:30.522-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 99 44617678156553 0 2016-04-07T03:39:30.532-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 100 43664774281249 0 2016-04-07T03:39:30.532-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 101 168256659396217 0 2016-04-07T03:39:30.532-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 102 72581164081667 0 2016-04-07T03:39:30.536-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 103 249780823072925 0 2016-04-07T03:39:30.547-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 104 37475425773863 0 2016-04-07T03:39:30.547-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 105 35019452408130 0 2016-04-07T03:39:30.547-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 106 18077123662528 0 2016-04-07T03:39:30.558-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 107 254957223940316 0 2016-04-07T03:39:30.558-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 108 144675191870330 0 2016-04-07T03:39:30.558-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 109 209864259102659 0 2016-04-07T03:39:30.561-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 110 89444224835498 0 2016-04-07T03:39:30.561-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 111 41469849729592 0 2016-04-07T03:39:30.577-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 113 37448075491195 0 2016-04-07T03:39:30.577-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 114 144947155080807 0 2016-04-07T03:39:30.578-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 112 171009751958818 0 2016-04-07T03:39:30.586-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 115 247033161399223 0 2016-04-07T03:39:30.586-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 116 172818124966926 0 2016-04-07T03:39:30.602-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 117 225051550579791 0 2016-04-07T03:39:30.602-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 118 152969137958763 0 2016-04-07T03:39:30.602-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 119 75675351256571 0 2016-04-07T03:39:30.602-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 120 38157108161838 0 2016-04-07T03:39:30.61-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:30.611-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 123 75460970506035 0 2016-04-07T03:39:30.611-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 121 61370980267321 0 2016-04-07T03:39:30.611-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 122 225408780379046 0 2016-04-07T03:39:30.623-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 125 44416797734006 0 2016-04-07T03:39:30.623-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 126 209842187795207 0 2016-04-07T03:39:30.623-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 124 184992205395847 0 2016-04-07T03:39:30.635-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 127 194936107989436 0 2016-04-07T03:39:30.635-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 128 48466688758310 0 2016-04-07T03:39:30.635-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 129 255563213160114 0 2016-04-07T03:39:30.635-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 130 179869284235957 0 2016-04-07T03:39:30.648-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 131 58847262152017 0 2016-04-07T03:39:30.648-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 132 65790664446672 0 2016-04-07T03:39:30.648-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 133 202089924515511 0 2016-04-07T03:39:30.648-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 134 258960912282623 0 2016-04-07T03:39:30.658-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 136 159910755686483 0 2016-04-07T03:39:30.658-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 137 52780730421411 0 2016-04-07T03:39:30.659-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 135 76732993543759 0 2016-04-07T03:39:30.66-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 138 90810704853422 0 2016-04-07T03:39:30.671-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 139 89512194880657 0 2016-04-07T03:39:30.671-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 140 13197641018599 0 2016-04-07T03:39:30.671-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 141 159094953027231 0 2016-04-07T03:39:30.683-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 142 124034217115248 0 2016-04-07T03:39:30.683-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 143 135848723334359 0 2016-04-07T03:39:30.683-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 144 68377332136479 0 2016-04-07T03:39:30.683-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 145 167350546720678 0 2016-04-07T03:39:30.698-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 146 233245012753986 0 2016-04-07T03:39:30.698-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 147 260112566181691 0 2016-04-07T03:39:30.698-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 148 101949627117438 0 2016-04-07T03:39:30.698-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 149 145766979956593 0 2016-04-07T03:39:30.711-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 150 154494232622834 0 2016-04-07T03:39:30.711-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 151 275298822627578 0 2016-04-07T03:39:30.724-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 152 91813308630122 0 2016-04-07T03:39:30.724-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 153 14891818501026 0 2016-04-07T03:39:30.748-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 154 66203730595960 0 2016-04-07T03:39:30.761-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 155 210204760082745 0 2016-04-07T03:39:30.761-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 156 61015683062402 0 2016-04-07T03:39:30.771-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 158 106946092662572 0 2016-04-07T03:39:30.771-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 159 16638325924971 0 2016-04-07T03:39:30.771-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 160 195694620445725 0 2016-04-07T03:39:30.771-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 157 149431967211174 0 2016-04-07T03:39:30.786-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 161 279269629599999 0 2016-04-07T03:39:30.786-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 162 17976651613776 0 2016-04-07T03:39:30.786-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 163 188222364463258 0 2016-04-07T03:39:30.796-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 166 49877626669033 0 2016-04-07T03:39:30.796-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 164 126995762553697 0 2016-04-07T03:39:30.796-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 165 160726196490179 0 2016-04-07T03:39:30.796-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 167 21736469686790 0 2016-04-07T03:39:30.806-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 168 12665373111507 0 2016-04-07T03:39:30.806-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 169 138629566165788 0 2016-04-07T03:39:30.806-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 170 60393084206152 0 2016-04-07T03:39:30.811-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 171 262706678065813 0 2016-04-07T03:39:30.811-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 172 277867948781218 0 2016-04-07T03:39:30.811-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:30.821-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 175 214273391948838 0 2016-04-07T03:39:30.821-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 173 235685783462628 0 2016-04-07T03:39:30.821-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 174 238288081842522 0 2016-04-07T03:39:30.833-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 177 116535207935052 0 2016-04-07T03:39:30.833-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 178 969732888564 0 2016-04-07T03:39:30.833-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 176 94940711059698 0 2016-04-07T03:39:30.833-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 179 126469956154590 0 2016-04-07T03:39:30.836-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 180 189899754877060 0 2016-04-07T03:39:30.846-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 183 143897599629439 0 2016-04-07T03:39:30.846-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 181 278910223852952 0 2016-04-07T03:39:30.847-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 182 120075682618586 0 2016-04-07T03:39:30.858-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 184 101877051371623 0 2016-04-07T03:39:30.858-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 185 56561433709955 0 2016-04-07T03:39:30.858-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 186 224066240410899 0 2016-04-07T03:39:30.874-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 188 142750784350809 0 2016-04-07T03:39:30.874-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 189 265322817884827 0 2016-04-07T03:39:30.874-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 190 238632877374317 0 2016-04-07T03:39:30.874-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 187 231729989627527 0 2016-04-07T03:39:30.886-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 192 71163454200094 0 2016-04-07T03:39:30.886-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 193 200033875885674 0 2016-04-07T03:39:30.886-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 191 20681353077794 0 2016-04-07T03:39:30.897-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 194 218088831853984 0 2016-04-07T03:39:30.897-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 195 260921650443910 0 2016-04-07T03:39:30.897-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 196 85117050793768 0 2016-04-07T03:39:30.908-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 197 18031290358274 0 2016-04-07T03:39:30.909-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 198 280356671033097 0 2016-04-07T03:39:30.909-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 199 174979417462597 0 2016-04-07T03:39:30.909-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 200 254001499624611 0 2016-04-07T03:39:30.91-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 201 89197687070747 0 2016-04-07T03:39:30.923-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 203 136403233017711 0 2016-04-07T03:39:30.923-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 204 277082966151719 0 2016-04-07T03:39:30.923-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 202 213137035531765 0 2016-04-07T03:39:30.936-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 208 273284074787539 0 2016-04-07T03:39:30.936-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 205 91274231344064 0 2016-04-07T03:39:30.936-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 206 122508175098479 0 2016-04-07T03:39:30.936-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 207 279213240976406 0 2016-04-07T03:39:30.936-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 209 173676374998843 0 2016-04-07T03:39:30.948-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 210 38154172213549 0 2016-04-07T03:39:30.948-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 211 277097204482777 0 2016-04-07T03:39:30.948-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 212 251535579522538 0 2016-04-07T03:39:30.948-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 213 203224462093754 0 2016-04-07T03:39:30.959-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 214 14000216585500 0 2016-04-07T03:39:30.959-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 215 33782976308575 0 2016-04-07T03:39:30.959-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 216 130719097800059 0 2016-04-07T03:39:30.959-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 217 197033134229524 0 2016-04-07T03:39:30.971-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 218 132235291062994 0 2016-04-07T03:39:30.971-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 219 96705154007407 0 2016-04-07T03:39:30.971-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 220 124781795537199 0 2016-04-07T03:39:30.971-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 221 167420551890059 0 2016-04-07T03:39:30.982-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 222 57735921979157 0 2016-04-07T03:39:30.982-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 223 209122955003352 0 2016-04-07T03:39:30.982-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 224 63134116679029 0 2016-04-07T03:39:30.986-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 225 266263189696655 0 2016-04-07T03:39:30.997-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 226 18351522552680 0 2016-04-07T03:39:30.997-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 227 229424308246626 0 2016-04-07T03:39:30.997-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 228 68082019608648 0 2016-04-07T03:39:31.007-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 229 260526566451753 0 2016-04-07T03:39:31.007-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 230 115228393453652 0 2016-04-07T03:39:31.007-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 231 249148688273577 0 2016-04-07T03:39:31.01-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:31.01-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 232 187151893369992 0 2016-04-07T03:39:31.022-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 233 265393874231346 0 2016-04-07T03:39:31.022-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 234 124869832128254 0 2016-04-07T03:39:31.022-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 235 78444589504214 0 2016-04-07T03:39:31.032-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 237 38483181025744 0 2016-04-07T03:39:31.032-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 238 83889122463519 0 2016-04-07T03:39:31.032-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 239 141976966132010 0 2016-04-07T03:39:31.032-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 236 74119585812274 0 2016-04-07T03:39:31.036-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 240 263921669242641 0 2016-04-07T03:39:31.047-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 241 265992145703823 0 2016-04-07T03:39:31.047-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 242 76718291393925 0 2016-04-07T03:39:31.047-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 243 17861344527307 0 2016-04-07T03:39:31.059-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 244 163812112834553 0 2016-04-07T03:39:31.059-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 245 114608797645810 0 2016-04-07T03:39:31.059-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 246 48609138585455 0 2016-04-07T03:39:31.059-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 247 251957739361270 0 2016-04-07T03:39:31.072-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 251 252774349242889 0 2016-04-07T03:39:31.073-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 248 186210147773983 0 2016-04-07T03:39:31.073-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 249 9677251097062 0 2016-04-07T03:39:31.073-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 250 233941328200458 0 2016-04-07T03:39:31.084-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 252 54590627270833 0 2016-04-07T03:39:31.084-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 253 22783249958085 0 2016-04-07T03:39:31.084-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 254 253784097064484 0 2016-04-07T03:39:31.085-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 255 95754746206862 0 2016-04-07T03:39:31.086-07:00 [Info] KVSender::sendMutationTopicRequest Success Projector 172.23.99.241:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae bucket-1 InstanceIds [9366861204008862690] 2016-04-07T03:39:31.086-07:00 [Info] KVSender::sendMutationTopicRequest Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae bucket-1 Instances [indexInstance: singlePartn: > ] 2016-04-07T03:39:31.125-07:00 [Info] DATP[->dataport ":9105"] new connection "172.23.99.245:39088" +2 2016-04-07T03:39:31.211-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:31.307-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10000 2016-04-07T03:39:31.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 258 200810722754504 0 2016-04-07T03:39:31.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 259 18522030095422 0 2016-04-07T03:39:31.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 260 45584125996312 0 2016-04-07T03:39:31.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 257 150967939893747 0 2016-04-07T03:39:31.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 261 272676185266833 0 2016-04-07T03:39:31.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 262 179793969187503 0 2016-04-07T03:39:31.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 263 222627704931754 0 2016-04-07T03:39:31.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 264 211498908391233 0 2016-04-07T03:39:31.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 265 230043342034686 0 2016-04-07T03:39:31.361-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 256 180516434017392 0 2016-04-07T03:39:31.387-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 266 2795274515143 0 2016-04-07T03:39:31.387-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 267 220708729905173 0 2016-04-07T03:39:31.387-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 268 85448393741112 0 2016-04-07T03:39:31.406-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 269 132677817070884 0 2016-04-07T03:39:31.406-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 270 50054022380315 0 2016-04-07T03:39:31.407-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 271 161026709174800 0 2016-04-07T03:39:31.407-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 272 194958846554708 0 2016-04-07T03:39:31.41-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:31.41-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 273 222930302953591 0 2016-04-07T03:39:31.432-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 274 153372552465571 0 2016-04-07T03:39:31.432-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 275 5085318607260 0 2016-04-07T03:39:31.446-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10000 QueuedCount 619 2016-04-07T03:39:31.452-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 278 193042227290874 0 2016-04-07T03:39:31.452-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 279 3709991211882 0 2016-04-07T03:39:31.452-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 280 134857785791324 0 2016-04-07T03:39:31.452-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 276 226430146231814 0 2016-04-07T03:39:31.452-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 277 34261619056150 0 2016-04-07T03:39:31.461-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 283 96328437197935 0 2016-04-07T03:39:31.461-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 281 88125266481170 0 2016-04-07T03:39:31.461-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 282 231621950183320 0 2016-04-07T03:39:31.472-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 284 76983822380266 0 2016-04-07T03:39:31.472-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 285 231159931512775 0 2016-04-07T03:39:31.472-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 286 163647856775038 0 2016-04-07T03:39:31.472-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 287 26346179176329 0 2016-04-07T03:39:31.472-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 288 267197721814401 0 2016-04-07T03:39:31.484-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 289 29951870968706 0 2016-04-07T03:39:31.484-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 290 60982304565716 0 2016-04-07T03:39:31.484-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 291 143421467342234 0 2016-04-07T03:39:31.484-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 292 173536073246915 0 2016-04-07T03:39:31.485-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 293 152192857158178 0 2016-04-07T03:39:31.497-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 294 19808413248158 0 2016-04-07T03:39:31.498-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 296 72184181589123 0 2016-04-07T03:39:31.498-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 297 238125611604548 0 2016-04-07T03:39:31.498-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 295 135318318508005 0 2016-04-07T03:39:31.509-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 298 109936306268106 0 2016-04-07T03:39:31.509-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 299 134742670655869 0 2016-04-07T03:39:31.509-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 300 119857343500066 0 2016-04-07T03:39:31.509-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 301 41669750899701 0 2016-04-07T03:39:31.526-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 302 190434058171374 0 2016-04-07T03:39:31.527-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 303 131086810440731 0 2016-04-07T03:39:31.527-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 305 49973672667429 0 2016-04-07T03:39:31.527-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 304 14135379027915 0 2016-04-07T03:39:31.536-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 306 243121593954679 0 2016-04-07T03:39:31.536-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 307 273925482385384 0 2016-04-07T03:39:31.536-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 308 142961291544147 0 2016-04-07T03:39:31.546-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 309 72298009771260 0 2016-04-07T03:39:31.546-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 310 120960066236279 0 2016-04-07T03:39:31.546-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 311 936143404010 0 2016-04-07T03:39:31.546-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 312 108989313865043 0 2016-04-07T03:39:31.556-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 313 126449987612192 0 2016-04-07T03:39:31.556-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 314 109763516547133 0 2016-04-07T03:39:31.556-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 315 176930282877523 0 2016-04-07T03:39:31.561-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 316 176842904908032 0 2016-04-07T03:39:31.561-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 317 86764159988736 0 2016-04-07T03:39:31.572-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 318 226219841703749 0 2016-04-07T03:39:31.572-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 319 191229716861354 0 2016-04-07T03:39:31.572-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 320 27659851546100 0 2016-04-07T03:39:31.572-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 321 261156426792401 0 2016-04-07T03:39:31.582-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 322 228981905275359 0 2016-04-07T03:39:31.582-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 323 8115179710851 0 2016-04-07T03:39:31.582-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 324 57094952896889 0 2016-04-07T03:39:31.582-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 325 75367439553154 0 2016-04-07T03:39:31.585-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 326 79770952596475 0 2016-04-07T03:39:31.596-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 327 229762450367782 0 2016-04-07T03:39:31.596-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 328 65374278995908 0 2016-04-07T03:39:31.596-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 329 135030674443989 0 2016-04-07T03:39:31.596-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 330 148460211415379 0 2016-04-07T03:39:31.607-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 331 234285206073881 0 2016-04-07T03:39:31.607-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 332 218350887170396 0 2016-04-07T03:39:31.623-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 333 7552004155130 0 2016-04-07T03:39:31.624-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 334 54456864897776 0 2016-04-07T03:39:31.624-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 335 250746352021088 0 2016-04-07T03:39:31.624-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 336 58221030012087 0 2016-04-07T03:39:31.625-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:31.636-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 339 47174442266944 0 2016-04-07T03:39:31.636-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 340 207065336398203 0 2016-04-07T03:39:31.636-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 337 50170282945424 0 2016-04-07T03:39:31.636-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 338 9213143851756 0 2016-04-07T03:39:31.646-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 343 10063353592220 0 2016-04-07T03:39:31.646-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 341 149207970195433 0 2016-04-07T03:39:31.646-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 342 205347867843856 0 2016-04-07T03:39:31.657-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 345 232583281458703 0 2016-04-07T03:39:31.657-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 346 190856910439541 0 2016-04-07T03:39:31.657-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 347 270755934767315 0 2016-04-07T03:39:31.657-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 344 53812351250442 0 2016-04-07T03:39:31.66-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 348 194068149586236 0 2016-04-07T03:39:31.673-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 349 74497944078972 0 2016-04-07T03:39:31.673-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 350 144578396642129 0 2016-04-07T03:39:31.673-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 351 218607612311392 0 2016-04-07T03:39:31.685-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 352 103543538079777 0 2016-04-07T03:39:31.685-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 353 262454207149024 0 2016-04-07T03:39:31.685-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 354 166827110775779 0 2016-04-07T03:39:31.698-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 356 12049573146744 0 2016-04-07T03:39:31.698-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 357 72659282342529 0 2016-04-07T03:39:31.699-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 355 175705993046959 0 2016-04-07T03:39:31.699-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20000 2016-04-07T03:39:31.71-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 358 281385660099360 0 2016-04-07T03:39:31.71-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 359 244550216360604 0 2016-04-07T03:39:31.71-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 360 32156150541385 0 2016-04-07T03:39:31.71-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 361 242463209069485 0 2016-04-07T03:39:31.726-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 362 61024626753647 0 2016-04-07T03:39:31.726-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 363 45170352746400 0 2016-04-07T03:39:31.726-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 364 262188359072514 0 2016-04-07T03:39:31.736-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 365 19346850453202 0 2016-04-07T03:39:31.736-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 366 159640544754468 0 2016-04-07T03:39:31.736-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 367 122909694818259 0 2016-04-07T03:39:31.751-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 369 259420887843993 0 2016-04-07T03:39:31.751-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 370 23483261592847 0 2016-04-07T03:39:31.751-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 371 77846214774929 0 2016-04-07T03:39:31.751-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 368 249973599224603 0 2016-04-07T03:39:31.761-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 372 162210079179405 0 2016-04-07T03:39:31.761-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 373 207682093229969 0 2016-04-07T03:39:31.78-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 377 17119179797562 0 2016-04-07T03:39:31.781-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 374 242273797951082 0 2016-04-07T03:39:31.781-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 375 83514541728754 0 2016-04-07T03:39:31.781-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 376 22113940046596 0 2016-04-07T03:39:31.786-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 378 225844935987076 0 2016-04-07T03:39:31.786-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 379 132680154886588 0 2016-04-07T03:39:31.796-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 380 281443330616047 0 2016-04-07T03:39:31.796-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 381 3888293507282 0 2016-04-07T03:39:31.796-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 382 61633752925032 0 2016-04-07T03:39:31.806-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 383 81824752200785 0 2016-04-07T03:39:31.806-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 384 136936507196140 0 2016-04-07T03:39:31.806-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 385 182959893550799 0 2016-04-07T03:39:31.806-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 386 68879124133635 0 2016-04-07T03:39:31.81-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:31.811-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 387 87407773942736 0 2016-04-07T03:39:31.811-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 388 269733093348292 0 2016-04-07T03:39:31.821-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 389 53039016439854 0 2016-04-07T03:39:31.821-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 390 109608691824085 0 2016-04-07T03:39:31.821-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 391 164892722490691 0 2016-04-07T03:39:31.831-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 394 203742913404508 0 2016-04-07T03:39:31.831-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 395 275014807722178 0 2016-04-07T03:39:31.832-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 392 164461862024013 0 2016-04-07T03:39:31.832-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 393 170635810485551 0 2016-04-07T03:39:31.836-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 396 108786845648939 0 2016-04-07T03:39:31.847-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 397 29154036841022 0 2016-04-07T03:39:31.857-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 400 4928760617251 0 2016-04-07T03:39:31.857-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 401 234377840149315 0 2016-04-07T03:39:31.857-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 398 253521046128183 0 2016-04-07T03:39:31.857-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 399 117533701966029 0 2016-04-07T03:39:31.86-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 402 232837240871058 0 2016-04-07T03:39:31.873-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 405 266136119919188 0 2016-04-07T03:39:31.873-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 406 274209576147843 0 2016-04-07T03:39:31.873-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 403 108313973195722 0 2016-04-07T03:39:31.873-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 404 79346653034895 0 2016-04-07T03:39:31.884-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 407 226889610276959 0 2016-04-07T03:39:31.884-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 408 117946442451573 0 2016-04-07T03:39:31.884-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 409 237935724675011 0 2016-04-07T03:39:31.884-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 410 29273632555873 0 2016-04-07T03:39:31.886-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 411 2905012946209 0 2016-04-07T03:39:31.896-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 412 214534637650329 0 2016-04-07T03:39:31.896-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 413 205587335974326 0 2016-04-07T03:39:31.896-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 414 41603402637227 0 2016-04-07T03:39:31.896-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 415 94524279120344 0 2016-04-07T03:39:31.897-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20000 QueuedCount 1402 2016-04-07T03:39:31.903-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 416 271940753624028 0 2016-04-07T03:39:31.903-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 417 225996273270996 0 2016-04-07T03:39:31.903-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 418 136156921797869 0 2016-04-07T03:39:31.903-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 419 53510209488146 0 2016-04-07T03:39:31.91-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 420 243840825312012 0 2016-04-07T03:39:31.91-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 421 72806115459256 0 2016-04-07T03:39:31.91-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 422 175516697269654 0 2016-04-07T03:39:31.91-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 423 205485893262317 0 2016-04-07T03:39:31.918-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 427 261025336232464 0 2016-04-07T03:39:31.919-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 428 246984636790207 0 2016-04-07T03:39:31.919-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 424 161636044641933 0 2016-04-07T03:39:31.919-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 425 214067788644085 0 2016-04-07T03:39:31.919-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 426 95711679533986 0 2016-04-07T03:39:31.927-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 429 63130087846346 0 2016-04-07T03:39:31.927-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 430 236642687671059 0 2016-04-07T03:39:31.927-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 431 62597597039059 0 2016-04-07T03:39:31.927-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 432 118649386431387 0 2016-04-07T03:39:31.934-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 436 46583077951403 0 2016-04-07T03:39:31.934-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 433 124352014222964 0 2016-04-07T03:39:31.934-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 434 249024093312104 0 2016-04-07T03:39:31.934-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 435 259348247941281 0 2016-04-07T03:39:31.935-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 437 77935844656092 0 2016-04-07T03:39:31.957-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 438 277281829509660 0 2016-04-07T03:39:31.957-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 439 26099627002015 0 2016-04-07T03:39:31.958-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 442 175342744363000 0 2016-04-07T03:39:31.958-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 443 5504053737473 0 2016-04-07T03:39:31.958-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 440 18314993731367 0 2016-04-07T03:39:31.958-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 441 168058038676282 0 2016-04-07T03:39:31.962-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 444 25603712800167 0 2016-04-07T03:39:31.962-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 445 186162120779085 0 2016-04-07T03:39:31.968-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 452 203431070679477 0 2016-04-07T03:39:31.969-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 447 209708805666881 0 2016-04-07T03:39:31.969-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 448 145818986347159 0 2016-04-07T03:39:31.969-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 449 153750031016210 0 2016-04-07T03:39:31.969-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 451 25836099060412 0 2016-04-07T03:39:31.969-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 446 63777019935378 0 2016-04-07T03:39:31.969-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 450 144300433199090 0 2016-04-07T03:39:31.978-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 453 20658630634177 0 2016-04-07T03:39:31.978-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 454 119486642407286 0 2016-04-07T03:39:31.978-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 455 163420053586021 0 2016-04-07T03:39:31.986-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 457 214140970533150 0 2016-04-07T03:39:31.986-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 458 2779838576134 0 2016-04-07T03:39:31.986-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 456 251248690396391 0 2016-04-07T03:39:31.995-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30000 2016-04-07T03:39:31.996-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 460 242411956670802 0 2016-04-07T03:39:31.996-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 461 257553744834661 0 2016-04-07T03:39:31.996-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 459 116286163340129 0 2016-04-07T03:39:32.005-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 464 151195464575283 0 2016-04-07T03:39:32.005-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 465 87757713996123 0 2016-04-07T03:39:32.007-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 462 125177483608506 0 2016-04-07T03:39:32.007-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 463 50191237845876 0 2016-04-07T03:39:32.011-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:32.011-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 466 57094075046146 0 2016-04-07T03:39:32.011-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 467 6167292896434 0 2016-04-07T03:39:32.02-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 468 188720125408896 0 2016-04-07T03:39:32.02-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 469 170606110826250 0 2016-04-07T03:39:32.02-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 470 244970601627086 0 2016-04-07T03:39:32.02-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 471 99376737569101 0 2016-04-07T03:39:32.028-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 472 63072379290926 0 2016-04-07T03:39:32.028-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 473 72459800269332 0 2016-04-07T03:39:32.028-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 474 275952478099730 0 2016-04-07T03:39:32.034-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 475 264913784169123 0 2016-04-07T03:39:32.034-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 476 255673121658903 0 2016-04-07T03:39:32.034-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 477 258226214350978 0 2016-04-07T03:39:32.034-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 478 247375813162202 0 2016-04-07T03:39:32.035-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 479 19991549616146 0 2016-04-07T03:39:32.043-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 480 123450316272988 0 2016-04-07T03:39:32.043-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 481 141284321936912 0 2016-04-07T03:39:32.043-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 482 158702573245226 0 2016-04-07T03:39:32.043-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 483 166976166202877 0 2016-04-07T03:39:32.05-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 484 187824826834778 0 2016-04-07T03:39:32.069-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 485 48623612700575 0 2016-04-07T03:39:32.071-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 486 61198915360185 0 2016-04-07T03:39:32.071-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 487 178234418319450 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 493 35710735488637 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 495 5443537061339 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 496 269419652453792 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 499 34014063808729 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 488 130469475961498 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 492 24573480580824 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 494 247137663372713 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 498 269872769241442 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 497 234985249549847 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 500 188480640218403 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 489 98690598177932 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 491 74856379905354 0 2016-04-07T03:39:32.075-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 490 143994035688052 0 2016-04-07T03:39:32.08-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 501 101268651754121 0 2016-04-07T03:39:32.08-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 502 57508461300141 0 2016-04-07T03:39:32.085-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 503 162937291880855 0 2016-04-07T03:39:32.085-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 504 280842501802127 0 2016-04-07T03:39:32.085-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 505 140518644763480 0 2016-04-07T03:39:32.093-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 507 126977418721711 0 2016-04-07T03:39:32.093-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 508 21251636946083 0 2016-04-07T03:39:32.093-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 509 105203272546084 0 2016-04-07T03:39:32.093-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 506 27210826455658 0 2016-04-07T03:39:32.096-07:00 [Info] KVSender::sendMutationTopicRequest Success Projector 172.23.99.245:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae bucket-1 InstanceIds [9366861204008862690] 2016-04-07T03:39:32.096-07:00 [Info] KVSender::sendMutationTopicRequest Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae bucket-1 Instances [indexInstance: singlePartn: > ] 2016-04-07T03:39:32.1-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 510 237266830039237 0 2016-04-07T03:39:32.1-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 511 101413900717833 0 2016-04-07T03:39:32.106-07:00 [Info] DATP[->dataport ":9105"] new connection "172.23.99.246:54392" +3 2016-04-07T03:39:32.211-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:32.225-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 40000 2016-04-07T03:39:32.252-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30000 QueuedCount 7010 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 516 132540418280880 0 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 518 170027015523272 0 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 519 248987987172556 0 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 521 32265052736499 0 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 522 248386340553288 0 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 514 235795344074209 0 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 513 189245290150054 0 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 515 260480675755081 0 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 517 77154515346601 0 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 520 235143781195207 0 2016-04-07T03:39:32.342-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 512 206544429742209 0 2016-04-07T03:39:32.367-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 523 105325356877852 0 2016-04-07T03:39:32.367-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 524 169074893188579 0 2016-04-07T03:39:32.367-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 525 179485009054757 0 2016-04-07T03:39:32.367-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 526 99290244142442 0 2016-04-07T03:39:32.391-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 529 186521543993923 0 2016-04-07T03:39:32.391-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 527 71633710548799 0 2016-04-07T03:39:32.391-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 528 47926086719567 0 2016-04-07T03:39:32.411-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:32.415-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 530 263046688638278 0 2016-04-07T03:39:32.415-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 531 143695742540127 0 2016-04-07T03:39:32.441-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 532 5324729080974 0 2016-04-07T03:39:32.441-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 533 843997831413 0 2016-04-07T03:39:32.453-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 50000 2016-04-07T03:39:32.463-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 534 2340387800204 0 2016-04-07T03:39:32.463-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 535 201267591806867 0 2016-04-07T03:39:32.48-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 40000 QueuedCount 5225 2016-04-07T03:39:32.488-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 536 78330382864598 0 2016-04-07T03:39:32.488-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 537 35978401219881 0 2016-04-07T03:39:32.488-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 538 143534650751719 0 2016-04-07T03:39:32.488-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 539 25880173378222 0 2016-04-07T03:39:32.491-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 540 110470873918138 0 2016-04-07T03:39:32.503-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 541 132134133189408 0 2016-04-07T03:39:32.503-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 542 151174514064988 0 2016-04-07T03:39:32.503-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 543 32803111875611 0 2016-04-07T03:39:32.503-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 544 266973322485476 0 2016-04-07T03:39:32.517-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 545 228334056081824 0 2016-04-07T03:39:32.517-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 546 149510528318849 0 2016-04-07T03:39:32.517-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 547 167200564706926 0 2016-04-07T03:39:32.528-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 548 135753384142523 0 2016-04-07T03:39:32.528-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 549 147084383364409 0 2016-04-07T03:39:32.528-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 550 107085394777814 0 2016-04-07T03:39:32.528-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 551 158481502169104 0 2016-04-07T03:39:32.528-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 552 171677261182702 0 2016-04-07T03:39:32.54-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 553 166509819154179 0 2016-04-07T03:39:32.54-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 554 189185330381149 0 2016-04-07T03:39:32.54-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 555 127483850952965 0 2016-04-07T03:39:32.54-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 556 87002764197716 0 2016-04-07T03:39:32.541-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 557 170212179040377 0 2016-04-07T03:39:32.553-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 559 252826243625906 0 2016-04-07T03:39:32.554-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 558 51677793023262 0 2016-04-07T03:39:32.554-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 560 22154852462201 0 2016-04-07T03:39:32.554-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 561 262741804030233 0 2016-04-07T03:39:32.565-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 563 224210776457211 0 2016-04-07T03:39:32.565-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 564 158084023183079 0 2016-04-07T03:39:32.565-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 562 71790300468342 0 2016-04-07T03:39:32.566-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 565 83514137772974 0 2016-04-07T03:39:32.578-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 569 67280154096210 0 2016-04-07T03:39:32.578-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 567 223610077545782 0 2016-04-07T03:39:32.578-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 568 107799527496462 0 2016-04-07T03:39:32.578-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 566 173434808009315 0 2016-04-07T03:39:32.589-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 571 16778274262318 0 2016-04-07T03:39:32.589-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 572 135712938819050 0 2016-04-07T03:39:32.589-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 570 67974962445854 0 2016-04-07T03:39:32.591-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 573 88968113157530 0 2016-04-07T03:39:32.602-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 576 46586332355931 0 2016-04-07T03:39:32.602-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 577 191094118263917 0 2016-04-07T03:39:32.603-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 574 91031177150237 0 2016-04-07T03:39:32.603-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 575 212125996506700 0 2016-04-07T03:39:32.611-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:32.613-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 579 57417892241565 0 2016-04-07T03:39:32.613-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 578 39688632577537 0 2016-04-07T03:39:32.613-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 580 118328006050331 0 2016-04-07T03:39:32.616-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 581 19105989474871 0 2016-04-07T03:39:32.628-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 582 202309015186980 0 2016-04-07T03:39:32.628-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 583 220179828206099 0 2016-04-07T03:39:32.628-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 584 261551018702437 0 2016-04-07T03:39:32.628-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 585 151193032631768 0 2016-04-07T03:39:32.642-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 586 224488649595832 0 2016-04-07T03:39:32.643-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 60000 2016-04-07T03:39:32.652-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 589 87609944430498 0 2016-04-07T03:39:32.652-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 587 115227545240145 0 2016-04-07T03:39:32.653-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 588 88167216036959 0 2016-04-07T03:39:32.662-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 593 168927310382174 0 2016-04-07T03:39:32.663-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 590 74430420654840 0 2016-04-07T03:39:32.663-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 591 102233610864864 0 2016-04-07T03:39:32.663-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 592 240814736758194 0 2016-04-07T03:39:32.666-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 594 127961356577124 0 2016-04-07T03:39:32.677-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 595 150313400920835 0 2016-04-07T03:39:32.677-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 596 163998561639769 0 2016-04-07T03:39:32.677-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 597 93321403465037 0 2016-04-07T03:39:32.677-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 598 162024154350047 0 2016-04-07T03:39:32.687-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 599 267459828166510 0 2016-04-07T03:39:32.687-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 600 255692508743549 0 2016-04-07T03:39:32.687-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 601 267437057819622 0 2016-04-07T03:39:32.687-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 602 252007048293645 0 2016-04-07T03:39:32.691-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 603 264165189508952 0 2016-04-07T03:39:32.702-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 606 144723333780253 0 2016-04-07T03:39:32.702-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 607 122192494783869 0 2016-04-07T03:39:32.702-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 604 186212821755687 0 2016-04-07T03:39:32.702-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 605 199996042984260 0 2016-04-07T03:39:32.713-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 608 263026960498967 0 2016-04-07T03:39:32.713-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 609 20005222458504 0 2016-04-07T03:39:32.713-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 610 84313320768611 0 2016-04-07T03:39:32.715-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 50000 QueuedCount 4586 2016-04-07T03:39:32.717-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 611 105554870682823 0 2016-04-07T03:39:32.728-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 612 104637957299273 0 2016-04-07T03:39:32.728-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 613 194381775852297 0 2016-04-07T03:39:32.728-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 614 89066566119651 0 2016-04-07T03:39:32.728-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 615 22810728345956 0 2016-04-07T03:39:32.738-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 616 172218801687448 0 2016-04-07T03:39:32.738-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 617 157600808531083 0 2016-04-07T03:39:32.738-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 618 97669258476891 0 2016-04-07T03:39:32.741-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 620 280457891924782 0 2016-04-07T03:39:32.741-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 619 192613374444164 0 2016-04-07T03:39:32.752-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 621 122782139305414 0 2016-04-07T03:39:32.752-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 622 10050908528946 0 2016-04-07T03:39:32.752-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 623 151071674044047 0 2016-04-07T03:39:32.763-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 624 83114496433647 0 2016-04-07T03:39:32.763-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 625 234290113918946 0 2016-04-07T03:39:32.763-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 626 149999021161708 0 2016-04-07T03:39:32.763-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 627 169349048479518 0 2016-04-07T03:39:32.767-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 628 79089898645614 0 2016-04-07T03:39:32.782-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 629 65989889841841 0 2016-04-07T03:39:32.782-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 630 91902341098024 0 2016-04-07T03:39:32.782-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 631 155595919504282 0 2016-04-07T03:39:32.792-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 634 74335387752488 0 2016-04-07T03:39:32.792-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 635 115112757761637 0 2016-04-07T03:39:32.792-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 632 136028123462583 0 2016-04-07T03:39:32.792-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 633 209037343212528 0 2016-04-07T03:39:32.802-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 637 66467919481663 0 2016-04-07T03:39:32.802-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 638 154382883417316 0 2016-04-07T03:39:32.802-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 639 28466910068085 0 2016-04-07T03:39:32.803-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 636 214457314652031 0 2016-04-07T03:39:32.82-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 640 168558323820148 0 2016-04-07T03:39:32.82-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 641 46297784529561 0 2016-04-07T03:39:32.82-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 642 99063760742380 0 2016-04-07T03:39:32.821-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 643 229204218079220 0 2016-04-07T03:39:32.821-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:32.829-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 644 102607837696652 0 2016-04-07T03:39:32.829-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 645 281418641197101 0 2016-04-07T03:39:32.829-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 646 64111749477319 0 2016-04-07T03:39:32.841-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 647 183739659604919 0 2016-04-07T03:39:32.841-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 648 32610216661753 0 2016-04-07T03:39:32.841-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 649 129054844608743 0 2016-04-07T03:39:32.841-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 650 65623209830326 0 2016-04-07T03:39:32.842-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 70000 2016-04-07T03:39:32.854-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 654 225133067928715 0 2016-04-07T03:39:32.854-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 655 29911930062591 0 2016-04-07T03:39:32.854-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 651 6787719549425 0 2016-04-07T03:39:32.854-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 652 148488195247704 0 2016-04-07T03:39:32.854-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 653 274575827407820 0 2016-04-07T03:39:32.866-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 657 8808735726301 0 2016-04-07T03:39:32.866-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 658 87238656422278 0 2016-04-07T03:39:32.866-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 659 239201095549644 0 2016-04-07T03:39:32.867-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 656 237123459875515 0 2016-04-07T03:39:32.878-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 663 168610889321216 0 2016-04-07T03:39:32.879-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 660 132004356482913 0 2016-04-07T03:39:32.879-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 661 252332786521182 0 2016-04-07T03:39:32.879-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 662 156848437074443 0 2016-04-07T03:39:32.904-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 664 160825688430686 0 2016-04-07T03:39:32.904-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 665 60412027976989 0 2016-04-07T03:39:32.904-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 666 128021762441460 0 2016-04-07T03:39:32.916-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 667 7333267177297 0 2016-04-07T03:39:32.916-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 668 199618467195849 0 2016-04-07T03:39:32.916-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 669 123555825120641 0 2016-04-07T03:39:32.93-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 670 46410314726043 0 2016-04-07T03:39:32.93-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 671 11286117868763 0 2016-04-07T03:39:32.93-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 672 278165405543644 0 2016-04-07T03:39:32.941-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 673 186250790003443 0 2016-04-07T03:39:32.941-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 674 152675542111687 0 2016-04-07T03:39:32.941-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 675 176419919173871 0 2016-04-07T03:39:32.941-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 676 120040517128662 0 2016-04-07T03:39:32.954-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 678 224623143080567 0 2016-04-07T03:39:32.954-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 679 209398368785540 0 2016-04-07T03:39:32.954-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 680 1082751378065 0 2016-04-07T03:39:32.955-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 677 66959380128351 0 2016-04-07T03:39:32.961-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 60000 QueuedCount 5982 2016-04-07T03:39:32.963-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 682 142502427796427 0 2016-04-07T03:39:32.963-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 683 61579452245381 0 2016-04-07T03:39:32.963-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 684 189259421060272 0 2016-04-07T03:39:32.963-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 681 92070855396408 0 2016-04-07T03:39:32.967-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 685 71823200546513 0 2016-04-07T03:39:32.967-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 686 132674326233091 0 2016-04-07T03:39:32.974-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 687 56251575679035 0 2016-04-07T03:39:32.974-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 688 79739850237455 0 2016-04-07T03:39:32.974-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 689 218037534259376 0 2016-04-07T03:39:32.974-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 690 240754732717185 0 2016-04-07T03:39:32.986-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 693 206707409170861 0 2016-04-07T03:39:32.986-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 694 125436250087297 0 2016-04-07T03:39:32.987-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 691 37297852521496 0 2016-04-07T03:39:32.987-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 692 218288896086325 0 2016-04-07T03:39:32.993-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 695 138904199450537 0 2016-04-07T03:39:32.993-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 696 65433089274768 0 2016-04-07T03:39:32.993-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 697 234250240469611 0 2016-04-07T03:39:32.999-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 698 105858360925237 0 2016-04-07T03:39:32.999-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 699 281069420762771 0 2016-04-07T03:39:32.999-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 700 163440636480652 0 2016-04-07T03:39:32.999-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 701 156538479194884 0 2016-04-07T03:39:33.003-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 80000 2016-04-07T03:39:33.006-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 702 71138628181503 0 2016-04-07T03:39:33.006-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 703 45264233209992 0 2016-04-07T03:39:33.006-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 704 243213826044937 0 2016-04-07T03:39:33.012-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 705 226325904954577 0 2016-04-07T03:39:33.012-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 706 163828535001498 0 2016-04-07T03:39:33.012-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 707 2770722310294 0 2016-04-07T03:39:33.012-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 708 32528018582630 0 2016-04-07T03:39:33.012-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:33.017-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 710 22760464128020 0 2016-04-07T03:39:33.017-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 711 26720682245290 0 2016-04-07T03:39:33.017-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 709 214050638232794 0 2016-04-07T03:39:33.024-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 715 41614135223352 0 2016-04-07T03:39:33.024-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 712 62680199611619 0 2016-04-07T03:39:33.024-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 713 267189094345660 0 2016-04-07T03:39:33.024-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 714 245628980642388 0 2016-04-07T03:39:33.031-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 717 103604213557668 0 2016-04-07T03:39:33.031-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 718 12368805863779 0 2016-04-07T03:39:33.031-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 716 189868324728513 0 2016-04-07T03:39:33.038-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 719 201801500639540 0 2016-04-07T03:39:33.038-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 720 60736828192534 0 2016-04-07T03:39:33.038-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 721 1662110906413 0 2016-04-07T03:39:33.038-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 722 8227994124533 0 2016-04-07T03:39:33.041-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 723 44547155196910 0 2016-04-07T03:39:33.041-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 724 78461002166584 0 2016-04-07T03:39:33.049-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 725 41827731005380 0 2016-04-07T03:39:33.049-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 726 111222154665448 0 2016-04-07T03:39:33.049-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 727 123537892483054 0 2016-04-07T03:39:33.056-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 728 71159534276001 0 2016-04-07T03:39:33.056-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 729 8174116097496 0 2016-04-07T03:39:33.056-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 730 38194558374288 0 2016-04-07T03:39:33.056-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 731 238674476514312 0 2016-04-07T03:39:33.063-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 734 32603673472280 0 2016-04-07T03:39:33.063-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 735 273979515391373 0 2016-04-07T03:39:33.063-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 732 84790510777350 0 2016-04-07T03:39:33.063-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 733 263807345988560 0 2016-04-07T03:39:33.067-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 736 5851861300448 0 2016-04-07T03:39:33.067-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 737 188436483644861 0 2016-04-07T03:39:33.074-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 738 187248610775909 0 2016-04-07T03:39:33.074-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 739 67929596919396 0 2016-04-07T03:39:33.074-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 740 194002465815309 0 2016-04-07T03:39:33.079-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 743 73924419285932 0 2016-04-07T03:39:33.079-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 741 45954394939999 0 2016-04-07T03:39:33.079-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 742 255027303379049 0 2016-04-07T03:39:33.084-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 744 210929263364388 0 2016-04-07T03:39:33.084-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 745 279727913163484 0 2016-04-07T03:39:33.084-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 746 142049505099413 0 2016-04-07T03:39:33.089-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 749 92265477208715 0 2016-04-07T03:39:33.09-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 747 255960645029658 0 2016-04-07T03:39:33.09-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 748 161106323762152 0 2016-04-07T03:39:33.091-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 750 151197350740116 0 2016-04-07T03:39:33.099-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 751 251423217796828 0 2016-04-07T03:39:33.099-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 752 205631694556851 0 2016-04-07T03:39:33.099-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 753 118686675914641 0 2016-04-07T03:39:33.099-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 754 48646856292388 0 2016-04-07T03:39:33.106-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 757 224410157219617 0 2016-04-07T03:39:33.106-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 755 161363217761317 0 2016-04-07T03:39:33.106-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 756 212653100203654 0 2016-04-07T03:39:33.113-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 758 20757937894359 0 2016-04-07T03:39:33.113-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 759 61099016703684 0 2016-04-07T03:39:33.113-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 760 229388024289486 0 2016-04-07T03:39:33.116-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 762 267098909799751 0 2016-04-07T03:39:33.117-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 761 228592118495191 0 2016-04-07T03:39:33.131-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 765 92562562517443 0 2016-04-07T03:39:33.131-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 766 216932220807362 0 2016-04-07T03:39:33.132-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 763 104593094527620 0 2016-04-07T03:39:33.132-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 764 171648185890950 0 2016-04-07T03:39:33.133-07:00 [Info] KVSender::sendMutationTopicRequest Success Projector 172.23.99.246:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae bucket-1 InstanceIds [9366861204008862690] 2016-04-07T03:39:33.133-07:00 [Info] KVSender::sendMutationTopicRequest Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae bucket-1 Instances [indexInstance: singlePartn: > ] 2016-04-07T03:39:33.139-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 767 257849663753769 0 2016-04-07T03:39:33.149-07:00 [Info] DATP[->dataport ":9105"] new connection "172.23.99.251:39098" +4 2016-04-07T03:39:33.159-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 90000 2016-04-07T03:39:33.203-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 70000 QueuedCount 6413 2016-04-07T03:39:33.212-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:33.299-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 100000 2016-04-07T03:39:33.382-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 775 117467884881867 0 2016-04-07T03:39:33.382-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 768 277427408668615 0 2016-04-07T03:39:33.382-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 769 244828682501946 0 2016-04-07T03:39:33.382-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 770 130469876747063 0 2016-04-07T03:39:33.382-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 771 17809131096668 0 2016-04-07T03:39:33.382-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 772 212134248569398 0 2016-04-07T03:39:33.382-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 773 213005267727481 0 2016-04-07T03:39:33.382-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 774 100227443262479 0 2016-04-07T03:39:33.407-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 777 128460892505135 0 2016-04-07T03:39:33.407-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 778 80102366184827 0 2016-04-07T03:39:33.407-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 776 118213365533230 0 2016-04-07T03:39:33.413-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:33.427-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 110000 2016-04-07T03:39:33.427-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 781 106737186035671 0 2016-04-07T03:39:33.428-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 779 11150859787912 0 2016-04-07T03:39:33.428-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 780 38640786839438 0 2016-04-07T03:39:33.431-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 782 78309936369528 0 2016-04-07T03:39:33.445-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 80000 QueuedCount 23831 2016-04-07T03:39:33.452-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 784 104134736163745 0 2016-04-07T03:39:33.453-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 783 13129385265383 0 2016-04-07T03:39:33.472-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 786 29533696808850 0 2016-04-07T03:39:33.472-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 785 38141439829270 0 2016-04-07T03:39:33.481-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 787 63624087316986 0 2016-04-07T03:39:33.481-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 788 129687172548530 0 2016-04-07T03:39:33.481-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 789 278910100246856 0 2016-04-07T03:39:33.481-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 790 109325762869967 0 2016-04-07T03:39:33.492-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 793 107741242835988 0 2016-04-07T03:39:33.492-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 794 88182673019997 0 2016-04-07T03:39:33.492-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 791 261538773065195 0 2016-04-07T03:39:33.492-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 792 148613749180755 0 2016-04-07T03:39:33.503-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 795 85124256407720 0 2016-04-07T03:39:33.503-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 796 145782931500407 0 2016-04-07T03:39:33.503-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 797 162516898246047 0 2016-04-07T03:39:33.503-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 798 136051911026339 0 2016-04-07T03:39:33.506-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 799 268804831243155 0 2016-04-07T03:39:33.519-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 800 70155122463112 0 2016-04-07T03:39:33.519-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 801 187716182210085 0 2016-04-07T03:39:33.519-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 802 102341453937 0 2016-04-07T03:39:33.519-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 803 131950401090492 0 2016-04-07T03:39:33.529-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 804 61233990663871 0 2016-04-07T03:39:33.529-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 805 256871625598086 0 2016-04-07T03:39:33.53-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 806 155551883333914 0 2016-04-07T03:39:33.531-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 807 84710407660418 0 2016-04-07T03:39:33.545-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 808 275229702481106 0 2016-04-07T03:39:33.545-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 809 86479580936727 0 2016-04-07T03:39:33.545-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 810 186953219343885 0 2016-04-07T03:39:33.551-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 120000 2016-04-07T03:39:33.556-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 813 205972829462798 0 2016-04-07T03:39:33.556-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 811 106488706001066 0 2016-04-07T03:39:33.556-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 812 249553719123054 0 2016-04-07T03:39:33.556-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 814 17576015919055 0 2016-04-07T03:39:33.566-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 817 24641238319709 0 2016-04-07T03:39:33.567-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 815 92941334449729 0 2016-04-07T03:39:33.567-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 816 160960127674643 0 2016-04-07T03:39:33.576-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 819 18222696768682 0 2016-04-07T03:39:33.578-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 820 93573857009092 0 2016-04-07T03:39:33.578-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 821 73782359848699 0 2016-04-07T03:39:33.578-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 818 223979709663688 0 2016-04-07T03:39:33.585-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 822 116832784336950 0 2016-04-07T03:39:33.592-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 823 265556993758314 0 2016-04-07T03:39:33.592-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 824 190543470100871 0 2016-04-07T03:39:33.592-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 825 65611868621161 0 2016-04-07T03:39:33.592-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 826 253984329639339 0 2016-04-07T03:39:33.603-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 828 179544835887772 0 2016-04-07T03:39:33.603-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 829 72427994080628 0 2016-04-07T03:39:33.603-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 830 274862906968797 0 2016-04-07T03:39:33.603-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 827 253764854046643 0 2016-04-07T03:39:33.606-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 831 257901743116286 0 2016-04-07T03:39:33.625-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 832 180132979933711 0 2016-04-07T03:39:33.625-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 833 41197510331256 0 2016-04-07T03:39:33.625-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 834 17852335767837 0 2016-04-07T03:39:33.628-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 835 97889742999189 0 2016-04-07T03:39:33.628-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 836 65719243264785 0 2016-04-07T03:39:33.628-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 837 236299412086712 0 2016-04-07T03:39:33.629-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:33.631-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 838 90662225320054 0 2016-04-07T03:39:33.647-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 839 53271314923669 0 2016-04-07T03:39:33.647-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 840 214337112792263 0 2016-04-07T03:39:33.647-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 841 253951338525548 0 2016-04-07T03:39:33.654-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 130000 2016-04-07T03:39:33.657-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 842 128793880172331 0 2016-04-07T03:39:33.657-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 843 70974446662613 0 2016-04-07T03:39:33.657-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 844 71499292531036 0 2016-04-07T03:39:33.658-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 90000 QueuedCount 30464 2016-04-07T03:39:33.669-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 847 131812881091751 0 2016-04-07T03:39:33.669-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 845 199661162117601 0 2016-04-07T03:39:33.669-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 846 42754149207391 0 2016-04-07T03:39:33.679-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 848 256850099901012 0 2016-04-07T03:39:33.679-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 849 183154265022649 0 2016-04-07T03:39:33.679-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 850 277406634023326 0 2016-04-07T03:39:33.679-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 851 160413109563573 0 2016-04-07T03:39:33.682-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 852 218179141153184 0 2016-04-07T03:39:33.692-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 855 239421652366760 0 2016-04-07T03:39:33.693-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 853 79653780614221 0 2016-04-07T03:39:33.693-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 854 44756174215838 0 2016-04-07T03:39:33.702-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 857 81170383289690 0 2016-04-07T03:39:33.702-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 858 242504594348261 0 2016-04-07T03:39:33.702-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 859 48339659981153 0 2016-04-07T03:39:33.703-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 856 46690451798882 0 2016-04-07T03:39:33.706-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 860 86612692172531 0 2016-04-07T03:39:33.717-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 861 199480807250160 0 2016-04-07T03:39:33.717-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 862 184023927417784 0 2016-04-07T03:39:33.717-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 863 62414349862034 0 2016-04-07T03:39:33.717-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 864 22484085057444 0 2016-04-07T03:39:33.729-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 868 158566531896003 0 2016-04-07T03:39:33.73-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 865 76690092068537 0 2016-04-07T03:39:33.73-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 866 77248945150809 0 2016-04-07T03:39:33.73-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 867 125666983752556 0 2016-04-07T03:39:33.731-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 869 265896900192742 0 2016-04-07T03:39:33.742-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 870 13894072011679 0 2016-04-07T03:39:33.742-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 871 230304331799376 0 2016-04-07T03:39:33.742-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 872 157724437244092 0 2016-04-07T03:39:33.742-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 873 15787990334973 0 2016-04-07T03:39:33.753-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 874 104843658350997 0 2016-04-07T03:39:33.753-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 875 136603764732945 0 2016-04-07T03:39:33.753-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 876 125383954139348 0 2016-04-07T03:39:33.757-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 877 155943229076045 0 2016-04-07T03:39:33.764-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 140000 2016-04-07T03:39:33.767-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 881 178400552104023 0 2016-04-07T03:39:33.768-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 878 28983590360174 0 2016-04-07T03:39:33.768-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 879 50092772541531 0 2016-04-07T03:39:33.768-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 880 256003371032934 0 2016-04-07T03:39:33.781-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 882 257981708475600 0 2016-04-07T03:39:33.782-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 883 155941682671516 0 2016-04-07T03:39:33.794-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 887 61707025590238 0 2016-04-07T03:39:33.794-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 884 218462841970998 0 2016-04-07T03:39:33.794-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 885 38729109183209 0 2016-04-07T03:39:33.794-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 886 193403301799215 0 2016-04-07T03:39:33.804-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 888 280005929598772 0 2016-04-07T03:39:33.804-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 889 55729035851319 0 2016-04-07T03:39:33.804-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 890 29263391506211 0 2016-04-07T03:39:33.804-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 891 65994890485532 0 2016-04-07T03:39:33.806-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 892 121356016656632 0 2016-04-07T03:39:33.815-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:33.817-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 893 272613320136999 0 2016-04-07T03:39:33.817-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 894 248567629249625 0 2016-04-07T03:39:33.817-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 895 230260537300257 0 2016-04-07T03:39:33.827-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 896 40836658119759 0 2016-04-07T03:39:33.827-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 897 26737457082906 0 2016-04-07T03:39:33.827-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 898 185419888872786 0 2016-04-07T03:39:33.827-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 899 128114243374911 0 2016-04-07T03:39:33.831-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 900 30635211408084 0 2016-04-07T03:39:33.842-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 901 230172938615381 0 2016-04-07T03:39:33.842-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 902 17073798489537 0 2016-04-07T03:39:33.842-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 903 184437773729024 0 2016-04-07T03:39:33.852-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 904 140627842555840 0 2016-04-07T03:39:33.852-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 905 40008455927399 0 2016-04-07T03:39:33.856-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 906 179969845094431 0 2016-04-07T03:39:33.856-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 907 219808944950040 0 2016-04-07T03:39:33.867-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 908 242170470019049 0 2016-04-07T03:39:33.867-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 909 266219682003157 0 2016-04-07T03:39:33.867-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 910 138776154984983 0 2016-04-07T03:39:33.867-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 911 260849100593655 0 2016-04-07T03:39:33.869-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 150000 2016-04-07T03:39:33.879-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 912 52945321756570 0 2016-04-07T03:39:33.879-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 913 13929066473223 0 2016-04-07T03:39:33.879-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 914 8699993390721 0 2016-04-07T03:39:33.881-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 915 72958368975961 0 2016-04-07T03:39:33.89-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 916 181389294137435 0 2016-04-07T03:39:33.89-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 917 94978444097628 0 2016-04-07T03:39:33.89-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 918 225444585656313 0 2016-04-07T03:39:33.897-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 921 231813374765966 0 2016-04-07T03:39:33.898-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 919 86158018065481 0 2016-04-07T03:39:33.898-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 920 220905885903533 0 2016-04-07T03:39:33.902-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 100000 QueuedCount 39204 2016-04-07T03:39:33.904-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 925 106877049587497 0 2016-04-07T03:39:33.905-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 922 242693675852897 0 2016-04-07T03:39:33.905-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 923 241742101605187 0 2016-04-07T03:39:33.905-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 924 89976325536713 0 2016-04-07T03:39:33.906-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 926 52395385047609 0 2016-04-07T03:39:33.914-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 927 119847530378205 0 2016-04-07T03:39:33.914-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 928 73434158109466 0 2016-04-07T03:39:33.914-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 929 10614889468010 0 2016-04-07T03:39:33.914-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 930 215680021475501 0 2016-04-07T03:39:33.92-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 934 245875901173628 0 2016-04-07T03:39:33.921-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 931 189912993884790 0 2016-04-07T03:39:33.921-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 932 141585327645875 0 2016-04-07T03:39:33.921-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 933 167550361722455 0 2016-04-07T03:39:33.928-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 935 66374731241349 0 2016-04-07T03:39:33.928-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 936 126283377261262 0 2016-04-07T03:39:33.928-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 937 160245099182813 0 2016-04-07T03:39:33.931-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 939 255690952830837 0 2016-04-07T03:39:33.931-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 938 270166890491132 0 2016-04-07T03:39:33.944-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 940 12254356662222 0 2016-04-07T03:39:33.944-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 941 126183997242702 0 2016-04-07T03:39:33.944-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 942 56677511479388 0 2016-04-07T03:39:33.944-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 943 79921698198544 0 2016-04-07T03:39:33.955-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 944 87908164318133 0 2016-04-07T03:39:33.955-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 945 225777131082958 0 2016-04-07T03:39:33.958-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 946 240909125958838 0 2016-04-07T03:39:33.959-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 947 119793962750489 0 2016-04-07T03:39:33.959-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 948 280225843488120 0 2016-04-07T03:39:33.963-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 952 216183144263549 0 2016-04-07T03:39:33.963-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 949 114164185856397 0 2016-04-07T03:39:33.963-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 950 171669101018675 0 2016-04-07T03:39:33.963-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 951 269043050018410 0 2016-04-07T03:39:33.969-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 953 236973811768799 0 2016-04-07T03:39:33.97-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 954 57902767340705 0 2016-04-07T03:39:33.97-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 955 246095463493841 0 2016-04-07T03:39:33.97-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 956 219073535443564 0 2016-04-07T03:39:33.973-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 160000 2016-04-07T03:39:33.976-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 957 155566399600265 0 2016-04-07T03:39:33.976-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 958 82228227961766 0 2016-04-07T03:39:33.976-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 959 96965691841152 0 2016-04-07T03:39:33.981-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 960 195402459062167 0 2016-04-07T03:39:33.981-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 961 109093154677654 0 2016-04-07T03:39:33.981-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 962 234437227275928 0 2016-04-07T03:39:33.992-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 964 262717032008256 0 2016-04-07T03:39:33.994-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 965 157624290085183 0 2016-04-07T03:39:33.994-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 963 165503633606078 0 2016-04-07T03:39:33.996-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 969 244965858821643 0 2016-04-07T03:39:33.996-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 966 98498643940412 0 2016-04-07T03:39:33.996-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 967 21258999113695 0 2016-04-07T03:39:33.996-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 968 157549444251377 0 2016-04-07T03:39:34.002-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 971 157796073844660 0 2016-04-07T03:39:34.002-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 972 169474213430352 0 2016-04-07T03:39:34.002-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 970 109642270939457 0 2016-04-07T03:39:34.006-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 973 217106554312069 0 2016-04-07T03:39:34.006-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 974 108815045821612 0 2016-04-07T03:39:34.013-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 975 271736365357256 0 2016-04-07T03:39:34.013-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 976 257071182395589 0 2016-04-07T03:39:34.013-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 977 191656789940632 0 2016-04-07T03:39:34.02-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 981 83440665381536 0 2016-04-07T03:39:34.021-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 978 168075212108200 0 2016-04-07T03:39:34.021-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 979 45244837927888 0 2016-04-07T03:39:34.021-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 980 5613050269665 0 2016-04-07T03:39:34.028-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 984 198765175797219 0 2016-04-07T03:39:34.028-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 985 242813519199002 0 2016-04-07T03:39:34.028-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 982 190418017067019 0 2016-04-07T03:39:34.028-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 983 10259966023094 0 2016-04-07T03:39:34.031-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 986 224075025893339 0 2016-04-07T03:39:34.038-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 987 16312088674925 0 2016-04-07T03:39:34.038-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 988 259754204134673 0 2016-04-07T03:39:34.038-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 989 14062492220941 0 2016-04-07T03:39:34.038-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 990 230940075988292 0 2016-04-07T03:39:34.046-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 991 83231355407050 0 2016-04-07T03:39:34.053-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 992 12850053496447 0 2016-04-07T03:39:34.053-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 993 39470212685497 0 2016-04-07T03:39:34.053-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 994 102862007991342 0 2016-04-07T03:39:34.056-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 995 73988747451571 0 2016-04-07T03:39:34.064-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 996 213784583698141 0 2016-04-07T03:39:34.064-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 997 266189931867073 0 2016-04-07T03:39:34.064-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 998 280980617259582 0 2016-04-07T03:39:34.064-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1000 89245539488007 0 2016-04-07T03:39:34.064-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 999 235840333744753 0 2016-04-07T03:39:34.064-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1001 73056933604881 0 2016-04-07T03:39:34.071-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1002 96926662203166 0 2016-04-07T03:39:34.071-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1003 259336286022955 0 2016-04-07T03:39:34.071-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1004 191705908178687 0 2016-04-07T03:39:34.072-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1005 136851889735643 0 2016-04-07T03:39:34.08-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1006 272494603914329 0 2016-04-07T03:39:34.08-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1007 102669803223520 0 2016-04-07T03:39:34.08-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1008 167697758838601 0 2016-04-07T03:39:34.08-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1009 89080050535614 0 2016-04-07T03:39:34.08-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1010 96707487162437 0 2016-04-07T03:39:34.081-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1011 102816301278365 0 2016-04-07T03:39:34.086-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 170000 2016-04-07T03:39:34.091-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1014 175281461728230 0 2016-04-07T03:39:34.091-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1015 102881355179657 0 2016-04-07T03:39:34.091-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1016 230355974590011 0 2016-04-07T03:39:34.091-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1012 16707016710547 0 2016-04-07T03:39:34.091-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1013 15457003132548 0 2016-04-07T03:39:34.098-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:34.1-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1017 169153928428703 0 2016-04-07T03:39:34.1-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1018 26702544073523 0 2016-04-07T03:39:34.1-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1019 16773367792707 0 2016-04-07T03:39:34.106-07:00 [Info] KVSender::sendMutationTopicRequest Success Projector 172.23.99.251:9999 Topic MAINT_STREAM_TOPIC_a0:85:3b:34:1d:f9:92:ae bucket-1 InstanceIds [9366861204008862690] 2016-04-07T03:39:34.107-07:00 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream MAINT_STREAM Bucket bucket-1. 2016-04-07T03:39:34.107-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1020 39471124844850 0 2016-04-07T03:39:34.107-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1021 143554878069271 0 2016-04-07T03:39:34.107-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1022 87008407393749 0 2016-04-07T03:39:34.107-07:00 [Info] TK StreamBegin MAINT_STREAM bucket-1 1023 75283402269187 0 2016-04-07T03:39:34.108-07:00 [Info] Indexer::handleStreamRequestDone StreamId MAINT_STREAM Bucket bucket-1 2016-04-07T03:39:34.108-07:00 [Info] Timekeeper::handleStreamRequestDone StreamId MAINT_STREAM Bucket bucket-1 2016-04-07T03:39:34.108-07:00 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM bucket bucket-1 2016-04-07T03:39:34.108-07:00 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM bucket bucket-1 2016-04-07T03:39:34.108-07:00 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2016-04-07T03:39:34.108-07:00 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2016-04-07T03:39:34.15-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 110000 QueuedCount 47545 2016-04-07T03:39:34.208-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 180000 2016-04-07T03:39:34.322-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 190000 2016-04-07T03:39:34.401-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 120000 QueuedCount 50000 2016-04-07T03:39:34.434-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 200000 2016-04-07T03:39:34.541-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 210000 2016-04-07T03:39:34.545-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:34.647-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 220000 2016-04-07T03:39:34.652-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 130000 QueuedCount 50000 2016-04-07T03:39:34.752-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 230000 2016-04-07T03:39:34.847-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 240000 2016-04-07T03:39:34.907-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 140000 QueuedCount 50000 2016-04-07T03:39:34.954-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 250000 2016-04-07T03:39:34.969-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:35.052-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 260000 2016-04-07T03:39:35.109-07:00 [Info] StreamState::adjustNonSnapAlignedVbs Bucket bucket-1 StreamId MAINT_STREAM Vbuckets []. 2016-04-07T03:39:35.109-07:00 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and Bucket bucket-1 2016-04-07T03:39:35.141-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 270000 2016-04-07T03:39:35.159-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 150000 QueuedCount 49993 2016-04-07T03:39:35.232-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 280000 2016-04-07T03:39:35.325-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 290000 2016-04-07T03:39:35.411-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 300000 2016-04-07T03:39:35.414-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 160000 QueuedCount 50000 2016-04-07T03:39:35.489-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 310000 2016-04-07T03:39:35.575-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 320000 2016-04-07T03:39:35.626-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 170000 QueuedCount 41072 2016-04-07T03:39:35.655-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 330000 2016-04-07T03:39:35.733-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 340000 2016-04-07T03:39:35.811-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 350000 2016-04-07T03:39:35.874-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 180000 QueuedCount 31072 2016-04-07T03:39:35.889-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 360000 2016-04-07T03:39:35.96-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 370000 2016-04-07T03:39:36.04-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 380000 2016-04-07T03:39:36.109-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 390000 2016-04-07T03:39:36.155-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 190000 QueuedCount 21072 2016-04-07T03:39:36.175-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 400000 2016-04-07T03:39:36.253-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 410000 2016-04-07T03:39:36.321-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 420000 2016-04-07T03:39:36.399-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 430000 2016-04-07T03:39:36.441-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 200000 QueuedCount 11072 2016-04-07T03:39:36.462-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 440000 2016-04-07T03:39:36.531-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 450000 2016-04-07T03:39:36.596-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 460000 2016-04-07T03:39:36.665-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 470000 2016-04-07T03:39:36.735-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 480000 2016-04-07T03:39:36.746-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 210000 QueuedCount 1072 2016-04-07T03:39:36.808-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 490000 2016-04-07T03:39:36.876-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 500000 2016-04-07T03:39:36.879-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.370065607s CommitTime 84.336132ms TotalFlushTime 5.021562621s TotalCommitTime 84.336132ms 2016-04-07T03:39:36.88-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 211072, 211072, 1 committed:true 2016-04-07T03:39:36.88-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 211072, 211072, 1 committed:true) SnapCreateDur 1.455337864s SnapOpenDur 114.664µs 2016-04-07T03:39:36.91-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:36.933-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 510000 2016-04-07T03:39:36.94-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:36.994-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 520000 2016-04-07T03:39:37.056-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 530000 2016-04-07T03:39:37.124-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 540000 2016-04-07T03:39:37.136-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 220000 QueuedCount 50000 2016-04-07T03:39:37.148-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:37.185-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 550000 2016-04-07T03:39:37.244-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 560000 2016-04-07T03:39:37.311-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 570000 2016-04-07T03:39:37.38-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 580000 2016-04-07T03:39:37.4-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 230000 QueuedCount 50000 2016-04-07T03:39:37.434-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 590000 2016-04-07T03:39:37.488-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 600000 2016-04-07T03:39:37.548-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 610000 2016-04-07T03:39:37.614-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 620000 2016-04-07T03:39:37.656-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 240000 QueuedCount 49951 2016-04-07T03:39:37.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 630000 2016-04-07T03:39:37.723-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:37.726-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 640000 2016-04-07T03:39:37.785-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 650000 2016-04-07T03:39:37.844-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 660000 2016-04-07T03:39:37.901-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 670000 2016-04-07T03:39:37.916-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 250000 QueuedCount 50000 2016-04-07T03:39:37.954-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 680000 2016-04-07T03:39:38.013-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 690000 2016-04-07T03:39:38.069-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 700000 2016-04-07T03:39:38.125-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 710000 2016-04-07T03:39:38.143-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 260000 QueuedCount 50000 2016-04-07T03:39:38.18-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 720000 2016-04-07T03:39:38.233-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 730000 2016-04-07T03:39:38.289-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 740000 2016-04-07T03:39:38.343-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 750000 2016-04-07T03:39:38.345-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:38.396-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 760000 2016-04-07T03:39:38.397-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 270000 QueuedCount 49934 2016-04-07T03:39:38.45-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 770000 2016-04-07T03:39:38.506-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 780000 2016-04-07T03:39:38.559-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 790000 2016-04-07T03:39:38.628-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 800000 2016-04-07T03:39:38.662-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 280000 QueuedCount 50000 2016-04-07T03:39:38.682-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 810000 2016-04-07T03:39:38.739-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 820000 2016-04-07T03:39:38.791-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 830000 2016-04-07T03:39:38.845-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 840000 2016-04-07T03:39:38.897-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 850000 2016-04-07T03:39:38.92-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 290000 QueuedCount 50000 2016-04-07T03:39:38.95-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 860000 2016-04-07T03:39:38.977-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:39.003-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 870000 2016-04-07T03:39:39.055-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 880000 2016-04-07T03:39:39.111-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 890000 2016-04-07T03:39:39.168-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 900000 2016-04-07T03:39:39.173-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 300000 QueuedCount 50000 2016-04-07T03:39:39.221-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 910000 2016-04-07T03:39:39.275-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 920000 2016-04-07T03:39:39.335-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 930000 2016-04-07T03:39:39.392-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 940000 2016-04-07T03:39:39.426-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 310000 QueuedCount 50000 2016-04-07T03:39:39.451-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 950000 2016-04-07T03:39:39.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 960000 2016-04-07T03:39:39.564-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 970000 2016-04-07T03:39:39.625-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 980000 2016-04-07T03:39:39.64-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:39.679-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 990000 2016-04-07T03:39:39.679-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 320000 QueuedCount 50000 2016-04-07T03:39:39.739-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1000000 2016-04-07T03:39:39.795-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1010000 2016-04-07T03:39:39.849-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1020000 2016-04-07T03:39:39.902-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1030000 2016-04-07T03:39:39.936-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 330000 QueuedCount 50000 2016-04-07T03:39:39.96-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1040000 2016-04-07T03:39:40.023-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1050000 2016-04-07T03:39:40.077-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1060000 2016-04-07T03:39:40.135-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1070000 2016-04-07T03:39:40.157-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 340000 QueuedCount 50000 2016-04-07T03:39:40.189-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1080000 2016-04-07T03:39:40.272-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1090000 2016-04-07T03:39:40.327-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1100000 2016-04-07T03:39:40.378-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:40.382-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1110000 2016-04-07T03:39:40.435-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1120000 2016-04-07T03:39:40.442-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 350000 QueuedCount 50000 2016-04-07T03:39:40.489-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1130000 2016-04-07T03:39:40.541-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1140000 2016-04-07T03:39:40.592-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1150000 2016-04-07T03:39:40.644-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1160000 2016-04-07T03:39:40.697-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1170000 2016-04-07T03:39:40.697-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 360000 QueuedCount 50000 2016-04-07T03:39:40.75-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1180000 2016-04-07T03:39:40.802-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1190000 2016-04-07T03:39:40.857-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1200000 2016-04-07T03:39:40.915-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1210000 2016-04-07T03:39:40.958-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 370000 QueuedCount 50000 2016-04-07T03:39:40.976-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1220000 2016-04-07T03:39:41.031-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1230000 2016-04-07T03:39:41.092-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1240000 2016-04-07T03:39:41.151-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1250000 2016-04-07T03:39:41.215-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1260000 2016-04-07T03:39:41.217-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 380000 QueuedCount 42618 2016-04-07T03:39:41.273-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1270000 2016-04-07T03:39:41.332-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1280000 2016-04-07T03:39:41.392-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1290000 2016-04-07T03:39:41.452-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1300000 2016-04-07T03:39:41.467-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 390000 QueuedCount 32618 2016-04-07T03:39:41.513-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1310000 2016-04-07T03:39:41.573-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1320000 2016-04-07T03:39:41.633-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1330000 2016-04-07T03:39:41.693-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1340000 2016-04-07T03:39:41.716-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 400000 QueuedCount 22618 2016-04-07T03:39:41.757-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1350000 2016-04-07T03:39:41.821-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1360000 2016-04-07T03:39:41.882-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1370000 2016-04-07T03:39:41.94-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1380000 2016-04-07T03:39:41.972-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 410000 QueuedCount 12618 2016-04-07T03:39:42.001-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1390000 2016-04-07T03:39:42.061-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1400000 2016-04-07T03:39:42.122-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1410000 2016-04-07T03:39:42.183-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1420000 2016-04-07T03:39:42.192-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 420000 QueuedCount 2618 2016-04-07T03:39:42.243-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1430000 2016-04-07T03:39:42.304-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1440000 2016-04-07T03:39:42.365-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1450000 2016-04-07T03:39:42.377-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.473µs CommitTime 91.908743ms TotalFlushTime 10.213048107s TotalCommitTime 176.244875ms 2016-04-07T03:39:42.377-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 422618, 422618, 2 committed:true 2016-04-07T03:39:42.377-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 422618, 422618, 2 committed:true) SnapCreateDur 94.607839ms SnapOpenDur 40.415µs 2016-04-07T03:39:42.411-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:42.414-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1460000 2016-04-07T03:39:42.477-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1470000 2016-04-07T03:39:42.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1480000 2016-04-07T03:39:42.552-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1490000 2016-04-07T03:39:42.604-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 430000 QueuedCount 50000 2016-04-07T03:39:42.605-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1500000 2016-04-07T03:39:42.66-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1510000 2016-04-07T03:39:42.702-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:42.718-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1520000 2016-04-07T03:39:42.769-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1530000 2016-04-07T03:39:42.823-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1540000 2016-04-07T03:39:42.872-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 440000 QueuedCount 50000 2016-04-07T03:39:42.876-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1550000 2016-04-07T03:39:42.932-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1560000 2016-04-07T03:39:42.946-07:00 [Info] CompactionDaemon: Checking fragmentation, IndexInstId: 9366861204008862690 Data:142575171, Disk:145305600, ExtraSnapshotData:0, Fragmentation:1.8790941298890063% 2016-04-07T03:39:42.984-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1570000 2016-04-07T03:39:43.039-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1580000 2016-04-07T03:39:43.092-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1590000 2016-04-07T03:39:43.142-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 450000 QueuedCount 50000 2016-04-07T03:39:43.148-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1600000 2016-04-07T03:39:43.203-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1610000 2016-04-07T03:39:43.258-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1620000 2016-04-07T03:39:43.311-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1630000 2016-04-07T03:39:43.367-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1640000 2016-04-07T03:39:43.407-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 460000 QueuedCount 50000 2016-04-07T03:39:43.419-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1650000 2016-04-07T03:39:43.469-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:43.474-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1660000 2016-04-07T03:39:43.528-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1670000 2016-04-07T03:39:43.582-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1680000 2016-04-07T03:39:43.637-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1690000 2016-04-07T03:39:43.67-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 470000 QueuedCount 50000 2016-04-07T03:39:43.691-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1700000 2016-04-07T03:39:43.744-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1710000 2016-04-07T03:39:43.799-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1720000 2016-04-07T03:39:43.854-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1730000 2016-04-07T03:39:43.91-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1740000 2016-04-07T03:39:43.937-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 480000 QueuedCount 50000 2016-04-07T03:39:43.966-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1750000 2016-04-07T03:39:44.02-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1760000 2016-04-07T03:39:44.074-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1770000 2016-04-07T03:39:44.132-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1780000 2016-04-07T03:39:44.165-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 490000 QueuedCount 50000 2016-04-07T03:39:44.185-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1790000 2016-04-07T03:39:44.24-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1800000 2016-04-07T03:39:44.295-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1810000 2016-04-07T03:39:44.309-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:44.352-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1820000 2016-04-07T03:39:44.407-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1830000 2016-04-07T03:39:44.428-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 500000 QueuedCount 50000 2016-04-07T03:39:44.462-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1840000 2016-04-07T03:39:44.515-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1850000 2016-04-07T03:39:44.569-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1860000 2016-04-07T03:39:44.624-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1870000 2016-04-07T03:39:44.679-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1880000 2016-04-07T03:39:44.691-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 510000 QueuedCount 50000 2016-04-07T03:39:44.733-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1890000 2016-04-07T03:39:44.788-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1900000 2016-04-07T03:39:44.844-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1910000 2016-04-07T03:39:44.899-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1920000 2016-04-07T03:39:44.954-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1930000 2016-04-07T03:39:44.957-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 520000 QueuedCount 50000 2016-04-07T03:39:45.011-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1940000 2016-04-07T03:39:45.065-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1950000 2016-04-07T03:39:45.108-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:45.125-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1960000 2016-04-07T03:39:45.178-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1970000 2016-04-07T03:39:45.219-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 530000 QueuedCount 50000 2016-04-07T03:39:45.232-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1980000 2016-04-07T03:39:45.287-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 1990000 2016-04-07T03:39:45.386-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2000000 2016-04-07T03:39:45.441-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2010000 2016-04-07T03:39:45.495-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2020000 2016-04-07T03:39:45.531-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 540000 QueuedCount 50000 2016-04-07T03:39:45.548-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2030000 2016-04-07T03:39:45.601-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2040000 2016-04-07T03:39:45.657-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2050000 2016-04-07T03:39:45.711-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2060000 2016-04-07T03:39:45.766-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2070000 2016-04-07T03:39:45.796-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 550000 QueuedCount 50000 2016-04-07T03:39:45.821-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2080000 2016-04-07T03:39:45.872-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2090000 2016-04-07T03:39:45.928-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2100000 2016-04-07T03:39:45.983-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2110000 2016-04-07T03:39:46.038-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2120000 2016-04-07T03:39:46.061-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 560000 QueuedCount 49988 2016-04-07T03:39:46.068-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:46.093-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2130000 2016-04-07T03:39:46.153-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2140000 2016-04-07T03:39:46.206-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2150000 2016-04-07T03:39:46.257-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2160000 2016-04-07T03:39:46.293-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 570000 QueuedCount 50000 2016-04-07T03:39:46.311-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2170000 2016-04-07T03:39:46.364-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2180000 2016-04-07T03:39:46.417-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2190000 2016-04-07T03:39:46.472-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2200000 2016-04-07T03:39:46.525-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2210000 2016-04-07T03:39:46.556-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 580000 QueuedCount 50000 2016-04-07T03:39:46.577-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2220000 2016-04-07T03:39:46.631-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2230000 2016-04-07T03:39:46.687-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2240000 2016-04-07T03:39:46.743-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2250000 2016-04-07T03:39:46.799-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2260000 2016-04-07T03:39:46.82-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 590000 QueuedCount 50000 2016-04-07T03:39:46.853-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2270000 2016-04-07T03:39:46.91-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2280000 2016-04-07T03:39:46.966-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2290000 2016-04-07T03:39:47.027-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2300000 2016-04-07T03:39:47.082-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 600000 QueuedCount 44109 2016-04-07T03:39:47.087-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2310000 2016-04-07T03:39:47.147-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2320000 2016-04-07T03:39:47.208-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2330000 2016-04-07T03:39:47.27-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2340000 2016-04-07T03:39:47.331-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2350000 2016-04-07T03:39:47.34-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 610000 QueuedCount 34109 2016-04-07T03:39:47.392-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2360000 2016-04-07T03:39:47.452-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2370000 2016-04-07T03:39:47.513-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2380000 2016-04-07T03:39:47.574-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2390000 2016-04-07T03:39:47.597-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 620000 QueuedCount 24109 2016-04-07T03:39:47.637-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2400000 2016-04-07T03:39:47.698-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2410000 2016-04-07T03:39:47.759-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2420000 2016-04-07T03:39:47.82-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2430000 2016-04-07T03:39:47.854-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 630000 QueuedCount 14109 2016-04-07T03:39:47.879-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2440000 2016-04-07T03:39:47.941-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2450000 2016-04-07T03:39:48.001-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2460000 2016-04-07T03:39:48.065-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2470000 2016-04-07T03:39:48.112-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 640000 QueuedCount 4109 2016-04-07T03:39:48.124-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2480000 2016-04-07T03:39:48.188-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2490000 2016-04-07T03:39:48.249-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2500000 2016-04-07T03:39:48.309-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2510000 2016-04-07T03:39:48.34-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.405µs CommitTime 108.130538ms TotalFlushTime 15.846770929s TotalCommitTime 284.375413ms 2016-04-07T03:39:48.34-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 644109, 644109, 3 committed:true 2016-04-07T03:39:48.34-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 644109, 644109, 3 committed:true) SnapCreateDur 113.020987ms SnapOpenDur 32.899µs 2016-04-07T03:39:48.37-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2520000 2016-04-07T03:39:48.377-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:48.403-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2530000 2016-04-07T03:39:48.438-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2540000 2016-04-07T03:39:48.475-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2550000 2016-04-07T03:39:48.504-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 650000 QueuedCount 50000 2016-04-07T03:39:48.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2560000 2016-04-07T03:39:48.55-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2570000 2016-04-07T03:39:48.598-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2580000 2016-04-07T03:39:48.652-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2590000 2016-04-07T03:39:48.709-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2600000 2016-04-07T03:39:48.762-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2610000 2016-04-07T03:39:48.779-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 660000 QueuedCount 50000 2016-04-07T03:39:48.817-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2620000 2016-04-07T03:39:48.871-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2630000 2016-04-07T03:39:48.927-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2640000 2016-04-07T03:39:48.938-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:48.982-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2650000 2016-04-07T03:39:49.038-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2660000 2016-04-07T03:39:49.05-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 670000 QueuedCount 50000 2016-04-07T03:39:49.093-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2670000 2016-04-07T03:39:49.147-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2680000 2016-04-07T03:39:49.201-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2690000 2016-04-07T03:39:49.256-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2700000 2016-04-07T03:39:49.368-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2710000 2016-04-07T03:39:49.371-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 680000 QueuedCount 50000 2016-04-07T03:39:49.424-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2720000 2016-04-07T03:39:49.479-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2730000 2016-04-07T03:39:49.534-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2740000 2016-04-07T03:39:49.588-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2750000 2016-04-07T03:39:49.642-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2760000 2016-04-07T03:39:49.649-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 690000 QueuedCount 50000 2016-04-07T03:39:49.695-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2770000 2016-04-07T03:39:49.748-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2780000 2016-04-07T03:39:49.803-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2790000 2016-04-07T03:39:49.857-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2800000 2016-04-07T03:39:49.911-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2810000 2016-04-07T03:39:49.915-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 700000 QueuedCount 50000 2016-04-07T03:39:49.965-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2820000 2016-04-07T03:39:49.967-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:50.021-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2830000 2016-04-07T03:39:50.076-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2840000 2016-04-07T03:39:50.13-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2850000 2016-04-07T03:39:50.184-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2860000 2016-04-07T03:39:50.185-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 710000 QueuedCount 50000 2016-04-07T03:39:50.239-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2870000 2016-04-07T03:39:50.292-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2880000 2016-04-07T03:39:50.346-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2890000 2016-04-07T03:39:50.402-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2900000 2016-04-07T03:39:50.451-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 720000 QueuedCount 50000 2016-04-07T03:39:50.454-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2910000 2016-04-07T03:39:50.507-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2920000 2016-04-07T03:39:50.561-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2930000 2016-04-07T03:39:50.614-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2940000 2016-04-07T03:39:50.668-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2950000 2016-04-07T03:39:50.685-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 730000 QueuedCount 50000 2016-04-07T03:39:50.723-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2960000 2016-04-07T03:39:50.777-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2970000 2016-04-07T03:39:50.831-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2980000 2016-04-07T03:39:50.884-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 2990000 2016-04-07T03:39:50.937-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3000000 2016-04-07T03:39:50.939-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:50.951-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 740000 QueuedCount 48630 2016-04-07T03:39:50.992-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3010000 2016-04-07T03:39:51.045-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3020000 2016-04-07T03:39:51.102-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3030000 2016-04-07T03:39:51.154-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3040000 2016-04-07T03:39:51.206-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3050000 2016-04-07T03:39:51.224-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 750000 QueuedCount 50000 2016-04-07T03:39:51.26-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3060000 2016-04-07T03:39:51.314-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3070000 2016-04-07T03:39:51.367-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3080000 2016-04-07T03:39:51.421-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3090000 2016-04-07T03:39:51.473-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3100000 2016-04-07T03:39:51.491-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 760000 QueuedCount 50000 2016-04-07T03:39:51.527-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3110000 2016-04-07T03:39:51.582-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3120000 2016-04-07T03:39:51.634-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3130000 2016-04-07T03:39:51.687-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3140000 2016-04-07T03:39:51.741-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3150000 2016-04-07T03:39:51.757-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 770000 QueuedCount 50000 2016-04-07T03:39:51.792-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3160000 2016-04-07T03:39:51.816-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:51.846-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3170000 2016-04-07T03:39:51.899-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3180000 2016-04-07T03:39:51.951-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3190000 2016-04-07T03:39:52.005-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3200000 2016-04-07T03:39:52.022-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 780000 QueuedCount 50000 2016-04-07T03:39:52.057-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3210000 2016-04-07T03:39:52.11-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3220000 2016-04-07T03:39:52.165-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3230000 2016-04-07T03:39:52.218-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3240000 2016-04-07T03:39:52.273-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3250000 2016-04-07T03:39:52.291-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 790000 QueuedCount 50000 2016-04-07T03:39:52.328-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3260000 2016-04-07T03:39:52.382-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3270000 2016-04-07T03:39:52.436-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3280000 2016-04-07T03:39:52.488-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3290000 2016-04-07T03:39:52.545-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3300000 2016-04-07T03:39:52.556-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 800000 QueuedCount 50000 2016-04-07T03:39:52.6-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3310000 2016-04-07T03:39:52.653-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3320000 2016-04-07T03:39:52.707-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3330000 2016-04-07T03:39:52.761-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3340000 2016-04-07T03:39:52.788-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 810000 QueuedCount 50000 2016-04-07T03:39:52.817-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3350000 2016-04-07T03:39:52.877-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3360000 2016-04-07T03:39:52.94-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3370000 2016-04-07T03:39:53-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3380000 2016-04-07T03:39:53.048-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 820000 QueuedCount 40485 2016-04-07T03:39:53.062-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3390000 2016-04-07T03:39:53.122-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3400000 2016-04-07T03:39:53.186-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3410000 2016-04-07T03:39:53.245-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3420000 2016-04-07T03:39:53.306-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3430000 2016-04-07T03:39:53.315-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 830000 QueuedCount 30485 2016-04-07T03:39:53.37-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3440000 2016-04-07T03:39:53.431-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3450000 2016-04-07T03:39:53.493-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3460000 2016-04-07T03:39:53.552-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3470000 2016-04-07T03:39:53.574-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 840000 QueuedCount 20485 2016-04-07T03:39:53.612-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3480000 2016-04-07T03:39:53.674-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3490000 2016-04-07T03:39:53.736-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3500000 2016-04-07T03:39:53.796-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3510000 2016-04-07T03:39:53.833-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 850000 QueuedCount 10485 2016-04-07T03:39:53.857-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3520000 2016-04-07T03:39:53.918-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3530000 2016-04-07T03:39:53.979-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3540000 2016-04-07T03:39:54.04-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3550000 2016-04-07T03:39:54.092-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 860000 QueuedCount 485 2016-04-07T03:39:54.101-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3560000 2016-04-07T03:39:54.162-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3570000 2016-04-07T03:39:54.224-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3580000 2016-04-07T03:39:54.239-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.845µs CommitTime 132.69838ms TotalFlushTime 21.39446678s TotalCommitTime 417.073793ms 2016-04-07T03:39:54.239-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 860485, 860485, 4 committed:true 2016-04-07T03:39:54.239-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 860485, 860485, 4 committed:true) SnapCreateDur 142.034081ms SnapOpenDur 40.937µs 2016-04-07T03:39:54.276-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:54.277-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3590000 2016-04-07T03:39:54.313-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3600000 2016-04-07T03:39:54.347-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3610000 2016-04-07T03:39:54.385-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3620000 2016-04-07T03:39:54.421-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3630000 2016-04-07T03:39:54.463-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3640000 2016-04-07T03:39:54.516-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3650000 2016-04-07T03:39:54.524-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 870000 QueuedCount 50000 2016-04-07T03:39:54.634-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3660000 2016-04-07T03:39:54.699-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3670000 2016-04-07T03:39:54.754-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3680000 2016-04-07T03:39:54.807-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3690000 2016-04-07T03:39:54.863-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3700000 2016-04-07T03:39:54.874-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 880000 QueuedCount 50000 2016-04-07T03:39:54.918-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3710000 2016-04-07T03:39:54.937-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:54.973-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3720000 2016-04-07T03:39:55.029-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3730000 2016-04-07T03:39:55.082-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3740000 2016-04-07T03:39:55.14-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3750000 2016-04-07T03:39:55.156-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 890000 QueuedCount 50000 2016-04-07T03:39:55.195-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3760000 2016-04-07T03:39:55.248-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3770000 2016-04-07T03:39:55.3-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3780000 2016-04-07T03:39:55.354-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3790000 2016-04-07T03:39:55.406-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3800000 2016-04-07T03:39:55.428-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 900000 QueuedCount 50000 2016-04-07T03:39:55.461-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3810000 2016-04-07T03:39:55.513-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3820000 2016-04-07T03:39:55.565-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3830000 2016-04-07T03:39:55.619-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3840000 2016-04-07T03:39:55.675-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3850000 2016-04-07T03:39:55.698-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 910000 QueuedCount 50000 2016-04-07T03:39:55.728-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3860000 2016-04-07T03:39:55.78-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3870000 2016-04-07T03:39:55.833-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3880000 2016-04-07T03:39:55.885-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3890000 2016-04-07T03:39:55.915-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:55.943-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3900000 2016-04-07T03:39:55.967-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 920000 QueuedCount 50000 2016-04-07T03:39:55.996-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3910000 2016-04-07T03:39:56.047-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3920000 2016-04-07T03:39:56.1-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3930000 2016-04-07T03:39:56.155-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3940000 2016-04-07T03:39:56.206-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 930000 QueuedCount 50000 2016-04-07T03:39:56.209-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3950000 2016-04-07T03:39:56.262-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3960000 2016-04-07T03:39:56.315-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3970000 2016-04-07T03:39:56.368-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3980000 2016-04-07T03:39:56.42-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 3990000 2016-04-07T03:39:56.473-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4000000 2016-04-07T03:39:56.475-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 940000 QueuedCount 50000 2016-04-07T03:39:56.527-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4010000 2016-04-07T03:39:56.582-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4020000 2016-04-07T03:39:56.635-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4030000 2016-04-07T03:39:56.689-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4040000 2016-04-07T03:39:56.744-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4050000 2016-04-07T03:39:56.744-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 950000 QueuedCount 50000 2016-04-07T03:39:56.797-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4060000 2016-04-07T03:39:56.804-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:56.85-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4070000 2016-04-07T03:39:56.902-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4080000 2016-04-07T03:39:56.955-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4090000 2016-04-07T03:39:57.009-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4100000 2016-04-07T03:39:57.015-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 960000 QueuedCount 50000 2016-04-07T03:39:57.066-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4110000 2016-04-07T03:39:57.119-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4120000 2016-04-07T03:39:57.175-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4130000 2016-04-07T03:39:57.236-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4140000 2016-04-07T03:39:57.29-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4150000 2016-04-07T03:39:57.29-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 970000 QueuedCount 50000 2016-04-07T03:39:57.347-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4160000 2016-04-07T03:39:57.402-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4170000 2016-04-07T03:39:57.469-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4180000 2016-04-07T03:39:57.522-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4190000 2016-04-07T03:39:57.56-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 980000 QueuedCount 50000 2016-04-07T03:39:57.575-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4200000 2016-04-07T03:39:57.628-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4210000 2016-04-07T03:39:57.684-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4220000 2016-04-07T03:39:57.74-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4230000 2016-04-07T03:39:57.802-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:39:57.803-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4240000 2016-04-07T03:39:57.867-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4250000 2016-04-07T03:39:57.869-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 990000 QueuedCount 50000 2016-04-07T03:39:57.925-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4260000 2016-04-07T03:39:57.981-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4270000 2016-04-07T03:39:58.037-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4280000 2016-04-07T03:39:58.094-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4290000 2016-04-07T03:39:58.156-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4300000 2016-04-07T03:39:58.216-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1000000 QueuedCount 50000 2016-04-07T03:39:58.219-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4310000 2016-04-07T03:39:58.279-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4320000 2016-04-07T03:39:58.338-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4330000 2016-04-07T03:39:58.394-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4340000 2016-04-07T03:39:58.452-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4350000 2016-04-07T03:39:58.514-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4360000 2016-04-07T03:39:58.526-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1010000 QueuedCount 50000 2016-04-07T03:39:58.575-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4370000 2016-04-07T03:39:58.636-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4380000 2016-04-07T03:39:58.695-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4390000 2016-04-07T03:39:58.752-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4400000 2016-04-07T03:39:58.81-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4410000 2016-04-07T03:39:58.871-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4420000 2016-04-07T03:39:58.871-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1020000 QueuedCount 50000 2016-04-07T03:39:58.934-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4430000 2016-04-07T03:39:58.996-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4440000 2016-04-07T03:39:59.056-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4450000 2016-04-07T03:39:59.126-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4460000 2016-04-07T03:39:59.196-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4470000 2016-04-07T03:39:59.225-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1030000 QueuedCount 43740 2016-04-07T03:39:59.262-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4480000 2016-04-07T03:39:59.332-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4490000 2016-04-07T03:39:59.402-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4500000 2016-04-07T03:39:59.47-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4510000 2016-04-07T03:39:59.538-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4520000 2016-04-07T03:39:59.573-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1040000 QueuedCount 33740 2016-04-07T03:39:59.604-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4530000 2016-04-07T03:39:59.677-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4540000 2016-04-07T03:39:59.746-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4550000 2016-04-07T03:39:59.816-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4560000 2016-04-07T03:39:59.882-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4570000 2016-04-07T03:39:59.943-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1050000 QueuedCount 23740 2016-04-07T03:39:59.952-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4580000 2016-04-07T03:40:00.019-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4590000 2016-04-07T03:40:00.088-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4600000 2016-04-07T03:40:00.159-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4610000 2016-04-07T03:40:00.227-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4620000 2016-04-07T03:40:00.292-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4630000 2016-04-07T03:40:00.303-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1060000 QueuedCount 13740 2016-04-07T03:40:00.361-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4640000 2016-04-07T03:40:00.429-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4650000 2016-04-07T03:40:00.497-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4660000 2016-04-07T03:40:00.565-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4670000 2016-04-07T03:40:00.633-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4680000 2016-04-07T03:40:00.659-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1070000 QueuedCount 3740 2016-04-07T03:40:00.7-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4690000 2016-04-07T03:40:00.768-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4700000 2016-04-07T03:40:00.834-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4710000 2016-04-07T03:40:00.895-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4720000 2016-04-07T03:40:00.957-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4730000 2016-04-07T03:40:00.959-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.75µs CommitTime 142.041018ms TotalFlushTime 27.73465721s TotalCommitTime 559.114811ms 2016-04-07T03:40:00.959-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 1073740, 1073740, 5 committed:true 2016-04-07T03:40:00.959-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 1073740, 1073740, 5 committed:true) SnapCreateDur 151.218629ms SnapOpenDur 50.41µs 2016-04-07T03:40:00.995-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:01.007-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4740000 2016-04-07T03:40:01.045-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4750000 2016-04-07T03:40:01.076-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4760000 2016-04-07T03:40:01.118-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4770000 2016-04-07T03:40:01.158-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4780000 2016-04-07T03:40:01.192-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1080000 QueuedCount 50000 2016-04-07T03:40:01.21-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4790000 2016-04-07T03:40:01.266-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4800000 2016-04-07T03:40:01.325-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4810000 2016-04-07T03:40:01.382-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4820000 2016-04-07T03:40:01.437-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4830000 2016-04-07T03:40:01.462-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1090000 QueuedCount 50000 2016-04-07T03:40:01.494-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4840000 2016-04-07T03:40:01.548-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4850000 2016-04-07T03:40:01.566-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:01.606-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4860000 2016-04-07T03:40:01.661-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4870000 2016-04-07T03:40:01.715-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4880000 2016-04-07T03:40:01.762-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1100000 QueuedCount 50000 2016-04-07T03:40:01.769-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4890000 2016-04-07T03:40:01.825-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4900000 2016-04-07T03:40:01.88-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4910000 2016-04-07T03:40:01.936-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4920000 2016-04-07T03:40:01.991-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4930000 2016-04-07T03:40:02.158-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4940000 2016-04-07T03:40:02.159-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1110000 QueuedCount 50000 2016-04-07T03:40:02.217-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4950000 2016-04-07T03:40:02.271-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4960000 2016-04-07T03:40:02.324-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4970000 2016-04-07T03:40:02.379-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4980000 2016-04-07T03:40:02.431-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 4990000 2016-04-07T03:40:02.477-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1120000 QueuedCount 50000 2016-04-07T03:40:02.485-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5000000 2016-04-07T03:40:02.539-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5010000 2016-04-07T03:40:02.593-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5020000 2016-04-07T03:40:02.644-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5030000 2016-04-07T03:40:02.698-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5040000 2016-04-07T03:40:02.752-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5050000 2016-04-07T03:40:02.781-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1130000 QueuedCount 50000 2016-04-07T03:40:02.789-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:02.806-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5060000 2016-04-07T03:40:02.86-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5070000 2016-04-07T03:40:02.915-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5080000 2016-04-07T03:40:02.971-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5090000 2016-04-07T03:40:03.027-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5100000 2016-04-07T03:40:03.08-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1140000 QueuedCount 50000 2016-04-07T03:40:03.082-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5110000 2016-04-07T03:40:03.137-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5120000 2016-04-07T03:40:03.191-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5130000 2016-04-07T03:40:03.247-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5140000 2016-04-07T03:40:03.3-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5150000 2016-04-07T03:40:03.358-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5160000 2016-04-07T03:40:03.381-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1150000 QueuedCount 50000 2016-04-07T03:40:03.412-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5170000 2016-04-07T03:40:03.466-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5180000 2016-04-07T03:40:03.518-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5190000 2016-04-07T03:40:03.571-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5200000 2016-04-07T03:40:03.624-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5210000 2016-04-07T03:40:03.675-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1160000 QueuedCount 49997 2016-04-07T03:40:03.678-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5220000 2016-04-07T03:40:03.732-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5230000 2016-04-07T03:40:03.786-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5240000 2016-04-07T03:40:03.838-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5250000 2016-04-07T03:40:03.858-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:03.893-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5260000 2016-04-07T03:40:03.93-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1170000 QueuedCount 50000 2016-04-07T03:40:03.947-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5270000 2016-04-07T03:40:03.999-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5280000 2016-04-07T03:40:04.051-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5290000 2016-04-07T03:40:04.105-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5300000 2016-04-07T03:40:04.159-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5310000 2016-04-07T03:40:04.214-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5320000 2016-04-07T03:40:04.223-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1180000 QueuedCount 50000 2016-04-07T03:40:04.268-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5330000 2016-04-07T03:40:04.323-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5340000 2016-04-07T03:40:04.376-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5350000 2016-04-07T03:40:04.427-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5360000 2016-04-07T03:40:04.48-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5370000 2016-04-07T03:40:04.52-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1190000 QueuedCount 50000 2016-04-07T03:40:04.533-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5380000 2016-04-07T03:40:04.585-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5390000 2016-04-07T03:40:04.639-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5400000 2016-04-07T03:40:04.692-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5410000 2016-04-07T03:40:04.747-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5420000 2016-04-07T03:40:04.802-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5430000 2016-04-07T03:40:04.815-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1200000 QueuedCount 50000 2016-04-07T03:40:04.856-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5440000 2016-04-07T03:40:04.916-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5450000 2016-04-07T03:40:04.974-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5460000 2016-04-07T03:40:05.033-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5470000 2016-04-07T03:40:05.092-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5480000 2016-04-07T03:40:05.105-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1210000 QueuedCount 40265 2016-04-07T03:40:05.15-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5490000 2016-04-07T03:40:05.208-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5500000 2016-04-07T03:40:05.267-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5510000 2016-04-07T03:40:05.323-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5520000 2016-04-07T03:40:05.383-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5530000 2016-04-07T03:40:05.39-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1220000 QueuedCount 30265 2016-04-07T03:40:05.446-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5540000 2016-04-07T03:40:05.503-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5550000 2016-04-07T03:40:05.561-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5560000 2016-04-07T03:40:05.617-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5570000 2016-04-07T03:40:05.676-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5580000 2016-04-07T03:40:05.68-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1230000 QueuedCount 20265 2016-04-07T03:40:05.736-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5590000 2016-04-07T03:40:05.796-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5600000 2016-04-07T03:40:05.855-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5610000 2016-04-07T03:40:05.915-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5620000 2016-04-07T03:40:05.97-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1240000 QueuedCount 10265 2016-04-07T03:40:05.974-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5630000 2016-04-07T03:40:06.031-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5640000 2016-04-07T03:40:06.089-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5650000 2016-04-07T03:40:06.149-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5660000 2016-04-07T03:40:06.209-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5670000 2016-04-07T03:40:06.262-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1250000 QueuedCount 265 2016-04-07T03:40:06.266-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5680000 2016-04-07T03:40:06.328-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5690000 2016-04-07T03:40:06.386-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5700000 2016-04-07T03:40:06.433-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.349µs CommitTime 153.073914ms TotalFlushTime 32.868336157s TotalCommitTime 712.188725ms 2016-04-07T03:40:06.433-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 1250265, 1250265, 6 committed:true 2016-04-07T03:40:06.433-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 1250265, 1250265, 6 committed:true) SnapCreateDur 164.824192ms SnapOpenDur 35.641µs 2016-04-07T03:40:06.448-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5710000 2016-04-07T03:40:06.466-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:06.483-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5720000 2016-04-07T03:40:06.517-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5730000 2016-04-07T03:40:06.553-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5740000 2016-04-07T03:40:06.589-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5750000 2016-04-07T03:40:06.628-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5760000 2016-04-07T03:40:06.703-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5770000 2016-04-07T03:40:06.769-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5780000 2016-04-07T03:40:06.778-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1260000 QueuedCount 50000 2016-04-07T03:40:06.832-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5790000 2016-04-07T03:40:06.889-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5800000 2016-04-07T03:40:06.955-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5810000 2016-04-07T03:40:07.017-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5820000 2016-04-07T03:40:07.022-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:07.124-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5830000 2016-04-07T03:40:07.151-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1270000 QueuedCount 50000 2016-04-07T03:40:07.187-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5840000 2016-04-07T03:40:07.257-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5850000 2016-04-07T03:40:07.317-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5860000 2016-04-07T03:40:07.375-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5870000 2016-04-07T03:40:07.435-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5880000 2016-04-07T03:40:07.493-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5890000 2016-04-07T03:40:07.512-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1280000 QueuedCount 50000 2016-04-07T03:40:07.546-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5900000 2016-04-07T03:40:07.598-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5910000 2016-04-07T03:40:07.663-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5920000 2016-04-07T03:40:07.72-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5930000 2016-04-07T03:40:07.78-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5940000 2016-04-07T03:40:07.81-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1290000 QueuedCount 50000 2016-04-07T03:40:07.836-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5950000 2016-04-07T03:40:07.888-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5960000 2016-04-07T03:40:07.942-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5970000 2016-04-07T03:40:08.025-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5980000 2016-04-07T03:40:08.084-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 5990000 2016-04-07T03:40:08.108-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:08.113-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1300000 QueuedCount 49500 2016-04-07T03:40:08.142-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6000000 2016-04-07T03:40:08.199-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6010000 2016-04-07T03:40:08.254-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6020000 2016-04-07T03:40:08.315-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6030000 2016-04-07T03:40:08.376-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6040000 2016-04-07T03:40:08.418-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1310000 QueuedCount 50000 2016-04-07T03:40:08.434-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6050000 2016-04-07T03:40:08.49-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6060000 2016-04-07T03:40:08.546-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6070000 2016-04-07T03:40:08.613-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6080000 2016-04-07T03:40:08.667-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6090000 2016-04-07T03:40:08.719-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1320000 QueuedCount 50000 2016-04-07T03:40:08.722-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6100000 2016-04-07T03:40:08.777-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6110000 2016-04-07T03:40:08.83-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6120000 2016-04-07T03:40:08.887-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6130000 2016-04-07T03:40:08.942-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6140000 2016-04-07T03:40:08.982-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1330000 QueuedCount 50000 2016-04-07T03:40:09.017-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6150000 2016-04-07T03:40:09.044-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:09.075-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6160000 2016-04-07T03:40:09.13-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6170000 2016-04-07T03:40:09.189-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6180000 2016-04-07T03:40:09.248-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6190000 2016-04-07T03:40:09.283-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1340000 QueuedCount 50000 2016-04-07T03:40:09.302-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6200000 2016-04-07T03:40:09.355-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6210000 2016-04-07T03:40:09.408-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6220000 2016-04-07T03:40:09.463-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6230000 2016-04-07T03:40:09.523-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6240000 2016-04-07T03:40:09.583-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6250000 2016-04-07T03:40:09.585-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1350000 QueuedCount 50000 2016-04-07T03:40:09.64-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6260000 2016-04-07T03:40:09.698-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6270000 2016-04-07T03:40:09.753-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6280000 2016-04-07T03:40:09.805-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6290000 2016-04-07T03:40:09.86-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6300000 2016-04-07T03:40:09.885-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1360000 QueuedCount 50000 2016-04-07T03:40:09.912-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6310000 2016-04-07T03:40:09.983-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6320000 2016-04-07T03:40:10.042-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6330000 2016-04-07T03:40:10.101-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6340000 2016-04-07T03:40:10.161-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6350000 2016-04-07T03:40:10.183-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1370000 QueuedCount 45149 2016-04-07T03:40:10.22-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6360000 2016-04-07T03:40:10.28-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6370000 2016-04-07T03:40:10.34-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6380000 2016-04-07T03:40:10.401-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6390000 2016-04-07T03:40:10.46-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6400000 2016-04-07T03:40:10.474-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1380000 QueuedCount 35149 2016-04-07T03:40:10.521-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6410000 2016-04-07T03:40:10.581-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6420000 2016-04-07T03:40:10.642-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6430000 2016-04-07T03:40:10.702-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6440000 2016-04-07T03:40:10.761-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6450000 2016-04-07T03:40:10.768-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1390000 QueuedCount 25149 2016-04-07T03:40:10.822-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6460000 2016-04-07T03:40:10.882-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6470000 2016-04-07T03:40:10.943-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6480000 2016-04-07T03:40:11.004-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6490000 2016-04-07T03:40:11.063-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1400000 QueuedCount 15149 2016-04-07T03:40:11.064-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6500000 2016-04-07T03:40:11.125-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6510000 2016-04-07T03:40:11.185-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6520000 2016-04-07T03:40:11.247-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6530000 2016-04-07T03:40:11.306-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6540000 2016-04-07T03:40:11.316-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1410000 QueuedCount 5149 2016-04-07T03:40:11.367-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6550000 2016-04-07T03:40:11.426-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6560000 2016-04-07T03:40:11.486-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6570000 2016-04-07T03:40:11.548-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6580000 2016-04-07T03:40:11.607-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6590000 2016-04-07T03:40:11.661-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.818µs CommitTime 159.956991ms TotalFlushTime 37.743428103s TotalCommitTime 872.145716ms 2016-04-07T03:40:11.661-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 1415149, 1415149, 7 committed:true 2016-04-07T03:40:11.661-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 1415149, 1415149, 7 committed:true) SnapCreateDur 172.993543ms SnapOpenDur 41.155µs 2016-04-07T03:40:11.668-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6600000 2016-04-07T03:40:11.694-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:11.707-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6610000 2016-04-07T03:40:11.74-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6620000 2016-04-07T03:40:11.773-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6630000 2016-04-07T03:40:11.81-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6640000 2016-04-07T03:40:11.834-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1420000 QueuedCount 50000 2016-04-07T03:40:11.851-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6650000 2016-04-07T03:40:11.903-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6660000 2016-04-07T03:40:11.957-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6670000 2016-04-07T03:40:12.019-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6680000 2016-04-07T03:40:12.08-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6690000 2016-04-07T03:40:12.137-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6700000 2016-04-07T03:40:12.156-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1430000 QueuedCount 50000 2016-04-07T03:40:12.19-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6710000 2016-04-07T03:40:12.377-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6720000 2016-04-07T03:40:12.454-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6730000 2016-04-07T03:40:12.514-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6740000 2016-04-07T03:40:12.572-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6750000 2016-04-07T03:40:12.578-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1440000 QueuedCount 50000 2016-04-07T03:40:12.58-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:12.634-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6760000 2016-04-07T03:40:12.688-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6770000 2016-04-07T03:40:12.742-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6780000 2016-04-07T03:40:12.798-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6790000 2016-04-07T03:40:12.854-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6800000 2016-04-07T03:40:12.899-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1450000 QueuedCount 50000 2016-04-07T03:40:12.911-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6810000 2016-04-07T03:40:12.94-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":5313321,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":5549342117,"bucket-1:myindex:avg_ts_items_count":180286,"bucket-1:myindex:build_progress":6,"bucket-1:myindex:data_size":449356252,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":572448768,"bucket-1:myindex:flush_queue_size":50032,"bucket-1:myindex:frag_percent":21,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":145043702,"bucket-1:myindex:items_count":1250265,"bucket-1:myindex:num_commits":7,"bucket-1:myindex:num_compactions":0,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":1382107,"bucket-1:myindex:num_docs_pending":13591218,"bucket-1:myindex:num_docs_queued":5026675,"bucket-1:myindex:num_flush_queued":1502053,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":7,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":5227892239,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"7 872145716 114054386414885118","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"1452022 6646081780 11031806371477272","bucket-1:myindex:timings/storage_handle_open":"7 169150 4164918460","bucket-1:myindex:timings/storage_info":"29 148943 1126718941","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"6 1332855 351116823477","bucket-1:myindex:timings/storage_meta_set":"7 2138685 761599237033","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"2904043 31431447799 957000477696508117","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":6815343,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":150,"memory_quota":31457280000,"memory_used":481665024,"needs_restart":false,"num_connections":0,"timings/stats_response":"1060 53430411 5327276527887"} 2016-04-07T03:40:12.947-07:00 [Info] CompactionDaemon: Checking fragmentation, IndexInstId: 9366861204008862690 Data:469193139, Disk:687783936, ExtraSnapshotData:0, Fragmentation:31.781899163169754% 2016-04-07T03:40:12.947-07:00 [Info] CompactionDaemon: Compacting index instance:9366861204008862690 2016-04-07T03:40:12.967-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6820000 2016-04-07T03:40:12.974-07:00 [Info] ForestDBSlice::Compact Compacting upto SeqNum 644109. Slice Id 0, IndexInstId 9366861204008862690, IndexDefnId 9366861204008862690 2016-04-07T03:40:13.024-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6830000 2016-04-07T03:40:13.081-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6840000 2016-04-07T03:40:13.135-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6850000 2016-04-07T03:40:13.193-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6860000 2016-04-07T03:40:13.249-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6870000 2016-04-07T03:40:13.302-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6880000 2016-04-07T03:40:13.344-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1460000 QueuedCount 49996 2016-04-07T03:40:13.356-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6890000 2016-04-07T03:40:13.412-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6900000 2016-04-07T03:40:13.468-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6910000 2016-04-07T03:40:13.519-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6920000 2016-04-07T03:40:13.574-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6930000 2016-04-07T03:40:13.628-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6940000 2016-04-07T03:40:13.678-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1470000 QueuedCount 49992 2016-04-07T03:40:13.684-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6950000 2016-04-07T03:40:13.739-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6960000 2016-04-07T03:40:13.792-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6970000 2016-04-07T03:40:13.845-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6980000 2016-04-07T03:40:13.88-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:13.9-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 6990000 2016-04-07T03:40:13.952-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7000000 2016-04-07T03:40:13.997-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1480000 QueuedCount 50000 2016-04-07T03:40:14.005-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7010000 2016-04-07T03:40:14.06-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7020000 2016-04-07T03:40:14.114-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7030000 2016-04-07T03:40:14.166-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7040000 2016-04-07T03:40:14.219-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7050000 2016-04-07T03:40:14.273-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7060000 2016-04-07T03:40:14.302-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1490000 QueuedCount 50000 2016-04-07T03:40:14.328-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7070000 2016-04-07T03:40:14.382-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7080000 2016-04-07T03:40:14.437-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7090000 2016-04-07T03:40:14.493-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7100000 2016-04-07T03:40:14.548-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7110000 2016-04-07T03:40:14.577-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1500000 QueuedCount 50000 2016-04-07T03:40:14.601-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7120000 2016-04-07T03:40:14.656-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7130000 2016-04-07T03:40:14.708-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7140000 2016-04-07T03:40:14.764-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7150000 2016-04-07T03:40:14.819-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7160000 2016-04-07T03:40:14.872-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7170000 2016-04-07T03:40:14.899-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1510000 QueuedCount 50000 2016-04-07T03:40:14.926-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7180000 2016-04-07T03:40:14.979-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7190000 2016-04-07T03:40:15.022-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:15.036-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7200000 2016-04-07T03:40:15.09-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7210000 2016-04-07T03:40:15.145-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7220000 2016-04-07T03:40:15.199-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7230000 2016-04-07T03:40:15.21-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1520000 QueuedCount 50000 2016-04-07T03:40:15.253-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7240000 2016-04-07T03:40:15.306-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7250000 2016-04-07T03:40:15.358-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7260000 2016-04-07T03:40:15.411-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7270000 2016-04-07T03:40:15.463-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7280000 2016-04-07T03:40:15.519-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7290000 2016-04-07T03:40:15.519-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1530000 QueuedCount 50000 2016-04-07T03:40:15.576-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7300000 2016-04-07T03:40:15.63-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7310000 2016-04-07T03:40:15.683-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7320000 2016-04-07T03:40:15.739-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7330000 2016-04-07T03:40:15.793-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7340000 2016-04-07T03:40:15.85-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7350000 2016-04-07T03:40:15.85-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1540000 QueuedCount 50000 2016-04-07T03:40:15.906-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7360000 2016-04-07T03:40:15.96-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7370000 2016-04-07T03:40:16.012-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7380000 2016-04-07T03:40:16.063-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7390000 2016-04-07T03:40:16.116-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7400000 2016-04-07T03:40:16.161-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1550000 QueuedCount 50000 2016-04-07T03:40:16.169-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7410000 2016-04-07T03:40:16.227-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7420000 2016-04-07T03:40:16.288-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7430000 2016-04-07T03:40:16.347-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7440000 2016-04-07T03:40:16.404-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7450000 2016-04-07T03:40:16.464-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1560000 QueuedCount 41079 2016-04-07T03:40:16.467-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7460000 2016-04-07T03:40:16.525-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7470000 2016-04-07T03:40:16.585-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7480000 2016-04-07T03:40:16.644-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7490000 2016-04-07T03:40:16.704-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7500000 2016-04-07T03:40:16.765-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7510000 2016-04-07T03:40:16.775-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1570000 QueuedCount 31079 2016-04-07T03:40:16.824-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7520000 2016-04-07T03:40:16.887-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7530000 2016-04-07T03:40:16.951-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7540000 2016-04-07T03:40:17.009-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7550000 2016-04-07T03:40:17.04-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1580000 QueuedCount 21079 2016-04-07T03:40:17.066-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7560000 2016-04-07T03:40:17.123-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7570000 2016-04-07T03:40:17.18-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7580000 2016-04-07T03:40:17.239-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7590000 2016-04-07T03:40:17.299-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7600000 2016-04-07T03:40:17.343-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1590000 QueuedCount 11079 2016-04-07T03:40:17.358-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7610000 2016-04-07T03:40:17.419-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7620000 2016-04-07T03:40:17.478-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7630000 2016-04-07T03:40:17.538-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7640000 2016-04-07T03:40:17.597-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7650000 2016-04-07T03:40:17.649-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1600000 QueuedCount 1079 2016-04-07T03:40:17.657-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7660000 2016-04-07T03:40:17.718-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7670000 2016-04-07T03:40:17.777-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7680000 2016-04-07T03:40:17.84-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7690000 2016-04-07T03:40:17.875-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.951µs CommitTime 151.341064ms TotalFlushTime 43.605103509s TotalCommitTime 1.02348678s 2016-04-07T03:40:17.875-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 1601079, 1601079, 8 committed:true 2016-04-07T03:40:17.875-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 1601079, 1601079, 8 committed:true) SnapCreateDur 162.095205ms SnapOpenDur 41.432µs 2016-04-07T03:40:17.902-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7700000 2016-04-07T03:40:17.912-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:17.936-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7710000 2016-04-07T03:40:17.97-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7720000 2016-04-07T03:40:18.004-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7730000 2016-04-07T03:40:18.044-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7740000 2016-04-07T03:40:18.082-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7750000 2016-04-07T03:40:18.133-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7760000 2016-04-07T03:40:18.191-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7770000 2016-04-07T03:40:18.192-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1610000 QueuedCount 50000 2016-04-07T03:40:18.245-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7780000 2016-04-07T03:40:18.298-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7790000 2016-04-07T03:40:18.349-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7800000 2016-04-07T03:40:18.404-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7810000 2016-04-07T03:40:18.458-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7820000 2016-04-07T03:40:18.513-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7830000 2016-04-07T03:40:18.525-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1620000 QueuedCount 49988 2016-04-07T03:40:18.568-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7840000 2016-04-07T03:40:18.619-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7850000 2016-04-07T03:40:18.654-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:18.676-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7860000 2016-04-07T03:40:18.732-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7870000 2016-04-07T03:40:18.795-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7880000 2016-04-07T03:40:18.852-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7890000 2016-04-07T03:40:18.856-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1630000 QueuedCount 50000 2016-04-07T03:40:18.908-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7900000 2016-04-07T03:40:18.962-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7910000 2016-04-07T03:40:19.015-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7920000 2016-04-07T03:40:19.069-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7930000 2016-04-07T03:40:19.125-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7940000 2016-04-07T03:40:19.133-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1640000 QueuedCount 50000 2016-04-07T03:40:19.183-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7950000 2016-04-07T03:40:19.24-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7960000 2016-04-07T03:40:19.297-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7970000 2016-04-07T03:40:19.353-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7980000 2016-04-07T03:40:19.406-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 7990000 2016-04-07T03:40:19.45-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1650000 QueuedCount 50000 2016-04-07T03:40:19.46-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8000000 2016-04-07T03:40:19.513-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8010000 2016-04-07T03:40:19.569-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8020000 2016-04-07T03:40:19.625-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8030000 2016-04-07T03:40:19.679-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8040000 2016-04-07T03:40:19.75-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8050000 2016-04-07T03:40:19.775-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1660000 QueuedCount 50000 2016-04-07T03:40:19.804-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8060000 2016-04-07T03:40:19.835-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:19.859-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8070000 2016-04-07T03:40:19.915-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8080000 2016-04-07T03:40:19.984-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8090000 2016-04-07T03:40:20.039-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8100000 2016-04-07T03:40:20.094-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8110000 2016-04-07T03:40:20.1-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1670000 QueuedCount 50000 2016-04-07T03:40:20.151-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8120000 2016-04-07T03:40:20.209-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8130000 2016-04-07T03:40:20.267-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8140000 2016-04-07T03:40:20.326-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8150000 2016-04-07T03:40:20.385-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8160000 2016-04-07T03:40:20.445-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8170000 2016-04-07T03:40:20.504-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8180000 2016-04-07T03:40:20.564-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8190000 2016-04-07T03:40:20.623-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8200000 2016-04-07T03:40:20.683-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8210000 2016-04-07T03:40:20.743-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8220000 2016-04-07T03:40:20.803-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8230000 2016-04-07T03:40:20.864-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8240000 2016-04-07T03:40:20.923-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8250000 2016-04-07T03:40:20.987-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8260000 2016-04-07T03:40:21.047-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8270000 2016-04-07T03:40:21.111-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8280000 2016-04-07T03:40:21.174-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8290000 2016-04-07T03:40:21.234-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8300000 2016-04-07T03:40:21.3-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8310000 2016-04-07T03:40:21.36-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8320000 2016-04-07T03:40:21.421-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8330000 2016-04-07T03:40:21.482-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8340000 2016-04-07T03:40:21.542-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8350000 2016-04-07T03:40:21.601-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8360000 2016-04-07T03:40:21.659-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8370000 2016-04-07T03:40:21.717-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8380000 2016-04-07T03:40:21.735-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1680000 QueuedCount 50000 2016-04-07T03:40:21.778-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8390000 2016-04-07T03:40:21.842-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8400000 2016-04-07T03:40:21.901-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8410000 2016-04-07T03:40:21.958-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8420000 2016-04-07T03:40:22.017-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8430000 2016-04-07T03:40:22.076-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8440000 2016-04-07T03:40:22.132-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8450000 2016-04-07T03:40:22.189-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8460000 2016-04-07T03:40:22.245-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8470000 2016-04-07T03:40:22.3-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8480000 2016-04-07T03:40:22.345-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1690000 QueuedCount 50000 2016-04-07T03:40:22.356-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8490000 2016-04-07T03:40:22.41-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8500000 2016-04-07T03:40:22.467-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8510000 2016-04-07T03:40:22.523-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8520000 2016-04-07T03:40:22.577-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8530000 2016-04-07T03:40:22.592-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:22.634-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8540000 2016-04-07T03:40:22.658-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1700000 QueuedCount 50000 2016-04-07T03:40:22.689-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8550000 2016-04-07T03:40:22.742-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8560000 2016-04-07T03:40:22.801-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8570000 2016-04-07T03:40:22.859-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8580000 2016-04-07T03:40:22.918-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8590000 2016-04-07T03:40:22.975-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8600000 2016-04-07T03:40:22.979-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1710000 QueuedCount 50000 2016-04-07T03:40:23.029-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8610000 2016-04-07T03:40:23.083-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8620000 2016-04-07T03:40:23.143-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8630000 2016-04-07T03:40:23.206-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8640000 2016-04-07T03:40:23.262-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8650000 2016-04-07T03:40:23.3-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1720000 QueuedCount 50000 2016-04-07T03:40:23.32-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8660000 2016-04-07T03:40:23.378-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8670000 2016-04-07T03:40:23.437-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8680000 2016-04-07T03:40:23.493-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8690000 2016-04-07T03:40:23.546-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8700000 2016-04-07T03:40:23.576-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1730000 QueuedCount 50000 2016-04-07T03:40:23.602-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8710000 2016-04-07T03:40:23.657-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8720000 2016-04-07T03:40:23.713-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8730000 2016-04-07T03:40:23.774-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8740000 2016-04-07T03:40:23.836-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8750000 2016-04-07T03:40:23.89-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1740000 QueuedCount 43816 2016-04-07T03:40:23.897-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8760000 2016-04-07T03:40:23.958-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8770000 2016-04-07T03:40:24.017-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8780000 2016-04-07T03:40:24.078-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8790000 2016-04-07T03:40:24.139-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8800000 2016-04-07T03:40:24.203-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8810000 2016-04-07T03:40:24.205-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1750000 QueuedCount 33816 2016-04-07T03:40:24.264-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8820000 2016-04-07T03:40:24.325-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8830000 2016-04-07T03:40:24.385-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8840000 2016-04-07T03:40:24.446-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8850000 2016-04-07T03:40:24.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8860000 2016-04-07T03:40:24.52-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1760000 QueuedCount 23816 2016-04-07T03:40:24.569-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8870000 2016-04-07T03:40:24.631-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8880000 2016-04-07T03:40:24.693-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8890000 2016-04-07T03:40:24.754-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8900000 2016-04-07T03:40:24.816-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8910000 2016-04-07T03:40:24.834-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1770000 QueuedCount 13816 2016-04-07T03:40:24.879-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8920000 2016-04-07T03:40:24.942-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8930000 2016-04-07T03:40:25.003-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8940000 2016-04-07T03:40:25.064-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8950000 2016-04-07T03:40:25.124-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8960000 2016-04-07T03:40:25.157-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1780000 QueuedCount 3816 2016-04-07T03:40:25.184-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8970000 2016-04-07T03:40:25.244-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8980000 2016-04-07T03:40:25.305-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 8990000 2016-04-07T03:40:25.367-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9000000 2016-04-07T03:40:25.428-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9010000 2016-04-07T03:40:25.456-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.683µs CommitTime 162.010569ms TotalFlushTime 50.824720618s TotalCommitTime 1.185497349s 2016-04-07T03:40:25.456-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 1783816, 1783816, 9 committed:true 2016-04-07T03:40:25.456-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 1783816, 1783816, 9 committed:true) SnapCreateDur 175.270719ms SnapOpenDur 40.746µs 2016-04-07T03:40:25.486-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9020000 2016-04-07T03:40:25.492-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:25.523-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9030000 2016-04-07T03:40:25.557-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9040000 2016-04-07T03:40:25.589-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9050000 2016-04-07T03:40:25.635-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9060000 2016-04-07T03:40:25.675-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9070000 2016-04-07T03:40:25.701-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1790000 QueuedCount 50000 2016-04-07T03:40:25.727-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9080000 2016-04-07T03:40:25.784-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9090000 2016-04-07T03:40:25.843-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9100000 2016-04-07T03:40:25.905-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9110000 2016-04-07T03:40:25.963-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9120000 2016-04-07T03:40:26.009-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1800000 QueuedCount 50000 2016-04-07T03:40:26.015-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9130000 2016-04-07T03:40:26.071-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9140000 2016-04-07T03:40:26.343-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9150000 2016-04-07T03:40:26.409-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9160000 2016-04-07T03:40:26.432-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:26.468-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9170000 2016-04-07T03:40:26.505-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1810000 QueuedCount 50000 2016-04-07T03:40:26.525-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9180000 2016-04-07T03:40:26.581-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9190000 2016-04-07T03:40:26.636-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9200000 2016-04-07T03:40:26.691-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9210000 2016-04-07T03:40:26.747-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9220000 2016-04-07T03:40:26.801-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9230000 2016-04-07T03:40:26.835-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1820000 QueuedCount 50000 2016-04-07T03:40:26.856-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9240000 2016-04-07T03:40:26.913-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9250000 2016-04-07T03:40:26.97-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9260000 2016-04-07T03:40:27.026-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9270000 2016-04-07T03:40:27.08-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9280000 2016-04-07T03:40:27.136-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9290000 2016-04-07T03:40:27.171-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1830000 QueuedCount 50000 2016-04-07T03:40:27.19-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9300000 2016-04-07T03:40:27.245-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9310000 2016-04-07T03:40:27.3-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9320000 2016-04-07T03:40:27.355-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9330000 2016-04-07T03:40:27.409-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9340000 2016-04-07T03:40:27.462-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9350000 2016-04-07T03:40:27.496-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1840000 QueuedCount 50000 2016-04-07T03:40:27.516-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9360000 2016-04-07T03:40:27.572-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9370000 2016-04-07T03:40:27.63-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:27.632-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9380000 2016-04-07T03:40:27.688-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9390000 2016-04-07T03:40:27.743-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9400000 2016-04-07T03:40:27.795-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9410000 2016-04-07T03:40:27.809-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1850000 QueuedCount 50000 2016-04-07T03:40:27.849-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9420000 2016-04-07T03:40:27.902-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9430000 2016-04-07T03:40:27.962-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9440000 2016-04-07T03:40:28.022-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9450000 2016-04-07T03:40:28.075-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9460000 2016-04-07T03:40:28.116-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1860000 QueuedCount 50000 2016-04-07T03:40:28.13-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9470000 2016-04-07T03:40:28.185-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9480000 2016-04-07T03:40:28.24-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9490000 2016-04-07T03:40:28.296-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9500000 2016-04-07T03:40:28.351-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9510000 2016-04-07T03:40:28.405-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9520000 2016-04-07T03:40:28.42-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1870000 QueuedCount 50000 2016-04-07T03:40:28.459-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9530000 2016-04-07T03:40:28.514-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9540000 2016-04-07T03:40:28.567-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9550000 2016-04-07T03:40:28.619-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9560000 2016-04-07T03:40:28.671-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9570000 2016-04-07T03:40:28.674-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1880000 QueuedCount 49834 2016-04-07T03:40:28.729-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9580000 2016-04-07T03:40:28.788-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9590000 2016-04-07T03:40:28.847-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9600000 2016-04-07T03:40:28.907-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9610000 2016-04-07T03:40:28.969-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9620000 2016-04-07T03:40:28.971-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1890000 QueuedCount 39834 2016-04-07T03:40:29.027-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9630000 2016-04-07T03:40:29.085-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9640000 2016-04-07T03:40:29.15-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9650000 2016-04-07T03:40:29.208-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9660000 2016-04-07T03:40:29.263-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1900000 QueuedCount 29834 2016-04-07T03:40:29.266-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9670000 2016-04-07T03:40:29.324-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9680000 2016-04-07T03:40:29.382-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9690000 2016-04-07T03:40:29.441-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9700000 2016-04-07T03:40:29.501-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9710000 2016-04-07T03:40:29.54-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1910000 QueuedCount 19834 2016-04-07T03:40:29.559-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9720000 2016-04-07T03:40:29.619-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9730000 2016-04-07T03:40:29.679-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9740000 2016-04-07T03:40:29.738-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9750000 2016-04-07T03:40:29.798-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9760000 2016-04-07T03:40:29.835-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1920000 QueuedCount 9834 2016-04-07T03:40:29.857-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9770000 2016-04-07T03:40:29.916-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9780000 2016-04-07T03:40:29.976-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9790000 2016-04-07T03:40:30.034-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9800000 2016-04-07T03:40:30.094-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9810000 2016-04-07T03:40:30.155-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9820000 2016-04-07T03:40:30.217-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9830000 2016-04-07T03:40:30.276-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9840000 2016-04-07T03:40:30.325-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.635µs CommitTime 189.459704ms TotalFlushTime 55.333425819s TotalCommitTime 1.374957053s 2016-04-07T03:40:30.325-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 1929834, 1929834, 10 committed:true 2016-04-07T03:40:30.325-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 1929834, 1929834, 10 committed:true) SnapCreateDur 202.288842ms SnapOpenDur 36.055µs 2016-04-07T03:40:30.335-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9850000 2016-04-07T03:40:30.348-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1930000 QueuedCount 6232 2016-04-07T03:40:30.361-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:30.374-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9860000 2016-04-07T03:40:30.41-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9870000 2016-04-07T03:40:30.443-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9880000 2016-04-07T03:40:30.49-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9890000 2016-04-07T03:40:30.543-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9900000 2016-04-07T03:40:30.597-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9910000 2016-04-07T03:40:30.66-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9920000 2016-04-07T03:40:30.696-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1940000 QueuedCount 50000 2016-04-07T03:40:30.714-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9930000 2016-04-07T03:40:30.767-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9940000 2016-04-07T03:40:30.827-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9950000 2016-04-07T03:40:30.882-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9960000 2016-04-07T03:40:30.894-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:30.937-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9970000 2016-04-07T03:40:30.991-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9980000 2016-04-07T03:40:31.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1950000 QueuedCount 49999 2016-04-07T03:40:31.047-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 9990000 2016-04-07T03:40:31.103-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10000000 2016-04-07T03:40:31.161-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10010000 2016-04-07T03:40:31.215-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10020000 2016-04-07T03:40:31.27-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10030000 2016-04-07T03:40:31.332-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10040000 2016-04-07T03:40:31.389-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10050000 2016-04-07T03:40:31.392-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1960000 QueuedCount 50000 2016-04-07T03:40:31.446-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10060000 2016-04-07T03:40:31.506-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10070000 2016-04-07T03:40:31.56-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10080000 2016-04-07T03:40:31.613-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10090000 2016-04-07T03:40:31.665-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10100000 2016-04-07T03:40:31.719-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1970000 QueuedCount 50000 2016-04-07T03:40:31.721-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10110000 2016-04-07T03:40:31.777-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10120000 2016-04-07T03:40:31.833-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10130000 2016-04-07T03:40:31.888-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10140000 2016-04-07T03:40:31.951-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10150000 2016-04-07T03:40:32.01-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10160000 2016-04-07T03:40:32.064-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1980000 QueuedCount 50000 2016-04-07T03:40:32.066-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10170000 2016-04-07T03:40:32.074-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:32.123-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10180000 2016-04-07T03:40:32.177-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10190000 2016-04-07T03:40:32.231-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10200000 2016-04-07T03:40:32.285-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10210000 2016-04-07T03:40:32.34-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10220000 2016-04-07T03:40:32.399-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 1990000 QueuedCount 50000 2016-04-07T03:40:32.4-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10230000 2016-04-07T03:40:32.456-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10240000 2016-04-07T03:40:32.509-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10250000 2016-04-07T03:40:32.561-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10260000 2016-04-07T03:40:32.613-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10270000 2016-04-07T03:40:32.668-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10280000 2016-04-07T03:40:32.722-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2000000 QueuedCount 50000 2016-04-07T03:40:32.724-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10290000 2016-04-07T03:40:32.779-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10300000 2016-04-07T03:40:32.833-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10310000 2016-04-07T03:40:32.888-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10320000 2016-04-07T03:40:32.946-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10330000 2016-04-07T03:40:33.003-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10340000 2016-04-07T03:40:33.059-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2010000 QueuedCount 50000 2016-04-07T03:40:33.061-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10350000 2016-04-07T03:40:33.115-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10360000 2016-04-07T03:40:33.169-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10370000 2016-04-07T03:40:33.222-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10380000 2016-04-07T03:40:33.281-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10390000 2016-04-07T03:40:33.342-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10400000 2016-04-07T03:40:33.4-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10410000 2016-04-07T03:40:33.47-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10420000 2016-04-07T03:40:33.529-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10430000 2016-04-07T03:40:33.587-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10440000 2016-04-07T03:40:33.645-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10450000 2016-04-07T03:40:33.706-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10460000 2016-04-07T03:40:33.765-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10470000 2016-04-07T03:40:33.823-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10480000 2016-04-07T03:40:33.88-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10490000 2016-04-07T03:40:33.939-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10500000 2016-04-07T03:40:33.998-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10510000 2016-04-07T03:40:34.057-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10520000 2016-04-07T03:40:34.068-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:34.116-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10530000 2016-04-07T03:40:34.175-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10540000 2016-04-07T03:40:34.233-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10550000 2016-04-07T03:40:34.291-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10560000 2016-04-07T03:40:34.35-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10570000 2016-04-07T03:40:34.41-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10580000 2016-04-07T03:40:34.468-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10590000 2016-04-07T03:40:34.529-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10600000 2016-04-07T03:40:34.59-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10610000 2016-04-07T03:40:34.619-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2020000 QueuedCount 50000 2016-04-07T03:40:34.646-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10620000 2016-04-07T03:40:34.708-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10630000 2016-04-07T03:40:34.769-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10640000 2016-04-07T03:40:34.831-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10650000 2016-04-07T03:40:34.893-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10660000 2016-04-07T03:40:34.952-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10670000 2016-04-07T03:40:34.952-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2030000 QueuedCount 50000 2016-04-07T03:40:35.012-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10680000 2016-04-07T03:40:35.073-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10690000 2016-04-07T03:40:35.137-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10700000 2016-04-07T03:40:35.197-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10710000 2016-04-07T03:40:35.254-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10720000 2016-04-07T03:40:35.312-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10730000 2016-04-07T03:40:35.321-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2040000 QueuedCount 50000 2016-04-07T03:40:35.368-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10740000 2016-04-07T03:40:35.421-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10750000 2016-04-07T03:40:35.476-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10760000 2016-04-07T03:40:35.531-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10770000 2016-04-07T03:40:35.585-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10780000 2016-04-07T03:40:35.642-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10790000 2016-04-07T03:40:35.648-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2050000 QueuedCount 50000 2016-04-07T03:40:35.702-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10800000 2016-04-07T03:40:35.756-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10810000 2016-04-07T03:40:35.814-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10820000 2016-04-07T03:40:35.874-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10830000 2016-04-07T03:40:35.935-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10840000 2016-04-07T03:40:35.973-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2060000 QueuedCount 44087 2016-04-07T03:40:35.993-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10850000 2016-04-07T03:40:36.058-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10860000 2016-04-07T03:40:36.119-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10870000 2016-04-07T03:40:36.179-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10880000 2016-04-07T03:40:36.24-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10890000 2016-04-07T03:40:36.291-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2070000 QueuedCount 34087 2016-04-07T03:40:36.301-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10900000 2016-04-07T03:40:36.364-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10910000 2016-04-07T03:40:36.423-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10920000 2016-04-07T03:40:36.487-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10930000 2016-04-07T03:40:36.55-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10940000 2016-04-07T03:40:36.604-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2080000 QueuedCount 24087 2016-04-07T03:40:36.611-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10950000 2016-04-07T03:40:36.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10960000 2016-04-07T03:40:36.735-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10970000 2016-04-07T03:40:36.802-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10980000 2016-04-07T03:40:36.863-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 10990000 2016-04-07T03:40:36.927-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2090000 QueuedCount 14087 2016-04-07T03:40:36.927-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11000000 2016-04-07T03:40:36.99-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11010000 2016-04-07T03:40:37.051-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11020000 2016-04-07T03:40:37.11-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11030000 2016-04-07T03:40:37.172-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11040000 2016-04-07T03:40:37.232-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11050000 2016-04-07T03:40:37.256-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2100000 QueuedCount 4087 2016-04-07T03:40:37.293-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11060000 2016-04-07T03:40:37.356-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11070000 2016-04-07T03:40:37.416-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11080000 2016-04-07T03:40:37.477-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11090000 2016-04-07T03:40:37.6-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.02µs CommitTime 197.987155ms TotalFlushTime 1m2.218020903s TotalCommitTime 1.572944208s 2016-04-07T03:40:37.6-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 2104087, 2104087, 11 committed:true 2016-04-07T03:40:37.6-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 2104087, 2104087, 11 committed:true) SnapCreateDur 209.121827ms SnapOpenDur 42.95µs 2016-04-07T03:40:37.64-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:37.654-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11100000 2016-04-07T03:40:37.693-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11110000 2016-04-07T03:40:37.726-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11120000 2016-04-07T03:40:37.763-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11130000 2016-04-07T03:40:37.803-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11140000 2016-04-07T03:40:37.815-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2110000 QueuedCount 50000 2016-04-07T03:40:38.137-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11150000 2016-04-07T03:40:38.195-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2120000 QueuedCount 50000 2016-04-07T03:40:38.491-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:38.559-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2130000 QueuedCount 50000 2016-04-07T03:40:38.65-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11160000 2016-04-07T03:40:38.931-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2140000 QueuedCount 50000 2016-04-07T03:40:39.106-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11170000 2016-04-07T03:40:39.314-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2150000 QueuedCount 50000 2016-04-07T03:40:39.562-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11180000 2016-04-07T03:40:39.69-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2160000 QueuedCount 50000 2016-04-07T03:40:39.904-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:40.051-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2170000 QueuedCount 50000 2016-04-07T03:40:40.075-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11190000 2016-04-07T03:40:40.41-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2180000 QueuedCount 50000 2016-04-07T03:40:40.707-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11200000 2016-04-07T03:40:40.714-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2190000 QueuedCount 50000 2016-04-07T03:40:41.067-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2200000 QueuedCount 50000 2016-04-07T03:40:41.406-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2210000 QueuedCount 41636 2016-04-07T03:40:41.731-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2220000 QueuedCount 31636 2016-04-07T03:40:42.036-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2230000 QueuedCount 21636 2016-04-07T03:40:42.323-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2240000 QueuedCount 11636 2016-04-07T03:40:42.606-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2250000 QueuedCount 1636 2016-04-07T03:40:42.877-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.633µs CommitTime 204.141792ms TotalFlushTime 1m7.053715365s TotalCommitTime 1.777086s 2016-04-07T03:40:42.877-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 2251636, 2251636, 12 committed:true 2016-04-07T03:40:42.877-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 2251636, 2251636, 12 committed:true) SnapCreateDur 215.619034ms SnapOpenDur 46.883µs 2016-04-07T03:40:42.905-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11210000 2016-04-07T03:40:42.913-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:42.94-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11220000 2016-04-07T03:40:42.978-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11230000 2016-04-07T03:40:43.009-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11240000 2016-04-07T03:40:43.047-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11250000 2016-04-07T03:40:43.082-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11260000 2016-04-07T03:40:43.118-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11270000 2016-04-07T03:40:43.154-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11280000 2016-04-07T03:40:43.215-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2260000 QueuedCount 50000 2016-04-07T03:40:43.34-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11290000 2016-04-07T03:40:43.54-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2270000 QueuedCount 50000 2016-04-07T03:40:43.604-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:43.912-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2280000 QueuedCount 50000 2016-04-07T03:40:44.157-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11300000 2016-04-07T03:40:44.276-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2290000 QueuedCount 50000 2016-04-07T03:40:44.678-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2300000 QueuedCount 50000 2016-04-07T03:40:44.761-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11310000 2016-04-07T03:40:44.824-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:45.042-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2310000 QueuedCount 50000 2016-04-07T03:40:45.337-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11320000 2016-04-07T03:40:45.397-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2320000 QueuedCount 50000 2016-04-07T03:40:45.76-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2330000 QueuedCount 50000 2016-04-07T03:40:45.985-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:46.113-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2340000 QueuedCount 50000 2016-04-07T03:40:46.467-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2350000 QueuedCount 50000 2016-04-07T03:40:46.768-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2360000 QueuedCount 50000 2016-04-07T03:40:47.108-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2370000 QueuedCount 49043 2016-04-07T03:40:47.435-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2380000 QueuedCount 39043 2016-04-07T03:40:47.767-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2390000 QueuedCount 29043 2016-04-07T03:40:48.094-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2400000 QueuedCount 19043 2016-04-07T03:40:48.429-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2410000 QueuedCount 9043 2016-04-07T03:40:49.071-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.635µs CommitTime 351.579893ms TotalFlushTime 1m12.621546751s TotalCommitTime 2.128665893s 2016-04-07T03:40:49.071-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 2419043, 2419043, 13 committed:true 2016-04-07T03:40:49.071-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 2419043, 2419043, 13 committed:true) SnapCreateDur 362.945711ms SnapOpenDur 56.078µs 2016-04-07T03:40:49.108-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2420000 QueuedCount 20854 2016-04-07T03:40:49.121-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:49.499-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2430000 QueuedCount 50000 2016-04-07T03:40:49.655-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:50.255-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2440000 QueuedCount 50000 2016-04-07T03:40:50.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 117 2016-04-07T03:40:50.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 117 2016-04-07T03:40:50.763-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2450000 QueuedCount 50000 2016-04-07T03:40:50.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 117 2016-04-07T03:40:50.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 117 2016-04-07T03:40:50.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 117 2016-04-07T03:40:50.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 117 2016-04-07T03:40:50.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 117 2016-04-07T03:40:50.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 117 2016-04-07T03:40:50.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 117 2016-04-07T03:40:50.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 117 2016-04-07T03:40:51.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 117 2016-04-07T03:40:51.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 117 2016-04-07T03:40:51.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 117 2016-04-07T03:40:51.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 117 2016-04-07T03:40:51.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 117 2016-04-07T03:40:51.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 117 2016-04-07T03:40:51.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 117 2016-04-07T03:40:51.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 117 2016-04-07T03:40:51.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 117 2016-04-07T03:40:51.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 117 2016-04-07T03:40:51.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 117 2016-04-07T03:40:51.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 117 2016-04-07T03:40:51.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 117 2016-04-07T03:40:51.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 117 2016-04-07T03:40:51.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 117 2016-04-07T03:40:51.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 117 2016-04-07T03:40:51.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 117 2016-04-07T03:40:51.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 117 2016-04-07T03:40:51.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 117 2016-04-07T03:40:51.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 117 2016-04-07T03:40:51.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 117 2016-04-07T03:40:51.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 117 2016-04-07T03:40:51.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 117 2016-04-07T03:40:51.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 117 2016-04-07T03:40:51.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 117 2016-04-07T03:40:51.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 117 2016-04-07T03:40:51.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 117 2016-04-07T03:40:51.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 117 2016-04-07T03:40:51.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 117 2016-04-07T03:40:51.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 117 2016-04-07T03:40:51.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 117 2016-04-07T03:40:51.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 117 2016-04-07T03:40:51.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 117 2016-04-07T03:40:51.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 117 2016-04-07T03:40:52.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 117 2016-04-07T03:40:52.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 117 2016-04-07T03:40:52.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 117 2016-04-07T03:40:52.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 117 2016-04-07T03:40:52.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 117 2016-04-07T03:40:52.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 117 2016-04-07T03:40:52.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 117 2016-04-07T03:40:52.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 117 2016-04-07T03:40:52.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 117 2016-04-07T03:40:52.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 117 2016-04-07T03:40:52.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 117 2016-04-07T03:40:52.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 117 2016-04-07T03:40:52.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 117 2016-04-07T03:40:52.397-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2460000 QueuedCount 50000 2016-04-07T03:40:52.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 117 2016-04-07T03:40:52.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 117 2016-04-07T03:40:52.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 117 2016-04-07T03:40:52.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 117 2016-04-07T03:40:52.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 117 2016-04-07T03:40:52.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 117 2016-04-07T03:40:52.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 117 2016-04-07T03:40:52.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 117 2016-04-07T03:40:52.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 117 2016-04-07T03:40:52.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 117 2016-04-07T03:40:52.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 117 2016-04-07T03:40:52.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 117 2016-04-07T03:40:52.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 117 2016-04-07T03:40:52.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 117 2016-04-07T03:40:52.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 117 2016-04-07T03:40:52.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 117 2016-04-07T03:40:52.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 117 2016-04-07T03:40:52.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 117 2016-04-07T03:40:52.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 117 2016-04-07T03:40:52.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 117 2016-04-07T03:40:53.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 117 2016-04-07T03:40:53.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 117 2016-04-07T03:40:53.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 117 2016-04-07T03:40:53.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 117 2016-04-07T03:40:53.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 117 2016-04-07T03:40:53.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 117 2016-04-07T03:40:53.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 117 2016-04-07T03:40:53.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 117 2016-04-07T03:40:53.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 117 2016-04-07T03:40:53.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 117 2016-04-07T03:40:53.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 117 2016-04-07T03:40:53.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 117 2016-04-07T03:40:53.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 117 2016-04-07T03:40:53.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 117 2016-04-07T03:40:53.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 117 2016-04-07T03:40:53.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 117 2016-04-07T03:40:53.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 117 2016-04-07T03:40:53.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 117 2016-04-07T03:40:53.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 117 2016-04-07T03:40:53.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 117 2016-04-07T03:40:53.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 117 2016-04-07T03:40:53.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 117 2016-04-07T03:40:53.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 117 2016-04-07T03:40:53.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 117 2016-04-07T03:40:53.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 117 2016-04-07T03:40:53.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 117 2016-04-07T03:40:53.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 117 2016-04-07T03:40:53.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 117 2016-04-07T03:40:53.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 117 2016-04-07T03:40:53.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 117 2016-04-07T03:40:53.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 117 2016-04-07T03:40:53.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 117 2016-04-07T03:40:53.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 117 2016-04-07T03:40:54.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 117 2016-04-07T03:40:54.009-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:54.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 117 2016-04-07T03:40:54.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 117 2016-04-07T03:40:54.086-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2470000 QueuedCount 50000 2016-04-07T03:40:54.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 117 2016-04-07T03:40:54.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 117 2016-04-07T03:40:54.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 117 2016-04-07T03:40:54.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 117 2016-04-07T03:40:54.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 117 2016-04-07T03:40:54.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 117 2016-04-07T03:40:54.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 117 2016-04-07T03:40:54.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 117 2016-04-07T03:40:54.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 117 2016-04-07T03:40:54.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 117 2016-04-07T03:40:54.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8700 Milliseconds Vbucket 117 2016-04-07T03:40:54.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8730 Milliseconds Vbucket 117 2016-04-07T03:40:54.435-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2480000 QueuedCount 50000 2016-04-07T03:40:54.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8760 Milliseconds Vbucket 117 2016-04-07T03:40:54.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8790 Milliseconds Vbucket 117 2016-04-07T03:40:54.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8820 Milliseconds Vbucket 117 2016-04-07T03:40:54.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8850 Milliseconds Vbucket 117 2016-04-07T03:40:54.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8880 Milliseconds Vbucket 117 2016-04-07T03:40:54.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8910 Milliseconds Vbucket 117 2016-04-07T03:40:54.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8940 Milliseconds Vbucket 117 2016-04-07T03:40:54.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8970 Milliseconds Vbucket 117 2016-04-07T03:40:54.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9000 Milliseconds Vbucket 117 2016-04-07T03:40:54.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9030 Milliseconds Vbucket 117 2016-04-07T03:40:54.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9060 Milliseconds Vbucket 117 2016-04-07T03:40:54.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9090 Milliseconds Vbucket 117 2016-04-07T03:40:54.794-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2490000 QueuedCount 50000 2016-04-07T03:40:54.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9120 Milliseconds Vbucket 117 2016-04-07T03:40:54.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9150 Milliseconds Vbucket 117 2016-04-07T03:40:54.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9180 Milliseconds Vbucket 117 2016-04-07T03:40:54.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9210 Milliseconds Vbucket 117 2016-04-07T03:40:54.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9240 Milliseconds Vbucket 117 2016-04-07T03:40:54.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9270 Milliseconds Vbucket 117 2016-04-07T03:40:54.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9300 Milliseconds Vbucket 117 2016-04-07T03:40:55.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9330 Milliseconds Vbucket 117 2016-04-07T03:40:55.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9360 Milliseconds Vbucket 117 2016-04-07T03:40:55.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9390 Milliseconds Vbucket 117 2016-04-07T03:40:55.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9420 Milliseconds Vbucket 117 2016-04-07T03:40:55.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9450 Milliseconds Vbucket 117 2016-04-07T03:40:55.145-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2500000 QueuedCount 50000 2016-04-07T03:40:55.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9480 Milliseconds Vbucket 117 2016-04-07T03:40:55.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9510 Milliseconds Vbucket 117 2016-04-07T03:40:55.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9540 Milliseconds Vbucket 117 2016-04-07T03:40:55.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9570 Milliseconds Vbucket 117 2016-04-07T03:40:55.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9600 Milliseconds Vbucket 117 2016-04-07T03:40:55.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9630 Milliseconds Vbucket 117 2016-04-07T03:40:55.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9660 Milliseconds Vbucket 117 2016-04-07T03:40:55.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9690 Milliseconds Vbucket 117 2016-04-07T03:40:55.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9720 Milliseconds Vbucket 117 2016-04-07T03:40:55.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9750 Milliseconds Vbucket 117 2016-04-07T03:40:55.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9780 Milliseconds Vbucket 117 2016-04-07T03:40:55.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9810 Milliseconds Vbucket 117 2016-04-07T03:40:55.523-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2510000 QueuedCount 50000 2016-04-07T03:40:55.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9840 Milliseconds Vbucket 117 2016-04-07T03:40:55.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9870 Milliseconds Vbucket 117 2016-04-07T03:40:55.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9900 Milliseconds Vbucket 117 2016-04-07T03:40:55.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9930 Milliseconds Vbucket 117 2016-04-07T03:40:55.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9960 Milliseconds Vbucket 117 2016-04-07T03:40:55.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9990 Milliseconds Vbucket 117 2016-04-07T03:40:55.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10020 Milliseconds Vbucket 117 2016-04-07T03:40:55.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10050 Milliseconds Vbucket 117 2016-04-07T03:40:55.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10080 Milliseconds Vbucket 117 2016-04-07T03:40:55.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10110 Milliseconds Vbucket 117 2016-04-07T03:40:55.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10140 Milliseconds Vbucket 117 2016-04-07T03:40:55.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10170 Milliseconds Vbucket 117 2016-04-07T03:40:55.873-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2520000 QueuedCount 50000 2016-04-07T03:40:55.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10200 Milliseconds Vbucket 117 2016-04-07T03:40:55.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10230 Milliseconds Vbucket 117 2016-04-07T03:40:55.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10260 Milliseconds Vbucket 117 2016-04-07T03:40:55.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10290 Milliseconds Vbucket 117 2016-04-07T03:40:56.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10320 Milliseconds Vbucket 117 2016-04-07T03:40:56.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10350 Milliseconds Vbucket 117 2016-04-07T03:40:56.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10380 Milliseconds Vbucket 117 2016-04-07T03:40:56.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10410 Milliseconds Vbucket 117 2016-04-07T03:40:56.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10440 Milliseconds Vbucket 117 2016-04-07T03:40:56.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10470 Milliseconds Vbucket 117 2016-04-07T03:40:56.193-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2530000 QueuedCount 40808 2016-04-07T03:40:56.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10500 Milliseconds Vbucket 117 2016-04-07T03:40:56.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10530 Milliseconds Vbucket 117 2016-04-07T03:40:56.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10560 Milliseconds Vbucket 117 2016-04-07T03:40:56.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10590 Milliseconds Vbucket 117 2016-04-07T03:40:56.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10620 Milliseconds Vbucket 117 2016-04-07T03:40:56.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10650 Milliseconds Vbucket 117 2016-04-07T03:40:56.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10680 Milliseconds Vbucket 117 2016-04-07T03:40:56.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10710 Milliseconds Vbucket 117 2016-04-07T03:40:56.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10740 Milliseconds Vbucket 117 2016-04-07T03:40:56.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10770 Milliseconds Vbucket 117 2016-04-07T03:40:56.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10800 Milliseconds Vbucket 117 2016-04-07T03:40:56.518-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2540000 QueuedCount 30808 2016-04-07T03:40:56.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10830 Milliseconds Vbucket 117 2016-04-07T03:40:56.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10860 Milliseconds Vbucket 117 2016-04-07T03:40:56.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10890 Milliseconds Vbucket 117 2016-04-07T03:40:56.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10920 Milliseconds Vbucket 117 2016-04-07T03:40:56.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10950 Milliseconds Vbucket 117 2016-04-07T03:40:56.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10980 Milliseconds Vbucket 117 2016-04-07T03:40:56.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11010 Milliseconds Vbucket 117 2016-04-07T03:40:56.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11040 Milliseconds Vbucket 117 2016-04-07T03:40:56.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11070 Milliseconds Vbucket 117 2016-04-07T03:40:56.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11100 Milliseconds Vbucket 117 2016-04-07T03:40:56.801-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2550000 QueuedCount 20808 2016-04-07T03:40:56.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11130 Milliseconds Vbucket 117 2016-04-07T03:40:56.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11160 Milliseconds Vbucket 117 2016-04-07T03:40:56.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11190 Milliseconds Vbucket 117 2016-04-07T03:40:56.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11220 Milliseconds Vbucket 117 2016-04-07T03:40:56.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11250 Milliseconds Vbucket 117 2016-04-07T03:40:56.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11280 Milliseconds Vbucket 117 2016-04-07T03:40:57.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11310 Milliseconds Vbucket 117 2016-04-07T03:40:57.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11340 Milliseconds Vbucket 117 2016-04-07T03:40:57.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11370 Milliseconds Vbucket 117 2016-04-07T03:40:57.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11400 Milliseconds Vbucket 117 2016-04-07T03:40:57.122-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2560000 QueuedCount 10808 2016-04-07T03:40:57.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11430 Milliseconds Vbucket 117 2016-04-07T03:40:57.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11460 Milliseconds Vbucket 117 2016-04-07T03:40:57.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11490 Milliseconds Vbucket 117 2016-04-07T03:40:57.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11520 Milliseconds Vbucket 117 2016-04-07T03:40:57.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11550 Milliseconds Vbucket 117 2016-04-07T03:40:57.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11580 Milliseconds Vbucket 117 2016-04-07T03:40:57.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11610 Milliseconds Vbucket 117 2016-04-07T03:40:57.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11640 Milliseconds Vbucket 117 2016-04-07T03:40:57.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11670 Milliseconds Vbucket 117 2016-04-07T03:40:57.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11700 Milliseconds Vbucket 117 2016-04-07T03:40:57.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11730 Milliseconds Vbucket 117 2016-04-07T03:40:57.448-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2570000 QueuedCount 808 2016-04-07T03:40:57.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11760 Milliseconds Vbucket 117 2016-04-07T03:40:57.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11790 Milliseconds Vbucket 117 2016-04-07T03:40:57.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11820 Milliseconds Vbucket 117 2016-04-07T03:40:57.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11850 Milliseconds Vbucket 117 2016-04-07T03:40:57.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11880 Milliseconds Vbucket 117 2016-04-07T03:40:57.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11910 Milliseconds Vbucket 117 2016-04-07T03:40:57.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11940 Milliseconds Vbucket 117 2016-04-07T03:40:57.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11970 Milliseconds Vbucket 117 2016-04-07T03:40:57.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12000 Milliseconds Vbucket 117 2016-04-07T03:40:57.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12030 Milliseconds Vbucket 117 2016-04-07T03:40:57.741-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.64µs CommitTime 227.500114ms TotalFlushTime 1m20.82588989s TotalCommitTime 2.356166007s 2016-04-07T03:40:57.741-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 2570808, 2570808, 14 committed:true 2016-04-07T03:40:57.741-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 2570808, 2570808, 14 committed:true) SnapCreateDur 238.095171ms SnapOpenDur 38.05µs 2016-04-07T03:40:57.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12060 Milliseconds Vbucket 117 2016-04-07T03:40:57.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12090 Milliseconds Vbucket 117 2016-04-07T03:40:57.786-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:57.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12120 Milliseconds Vbucket 117 2016-04-07T03:40:57.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12150 Milliseconds Vbucket 117 2016-04-07T03:40:57.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12180 Milliseconds Vbucket 117 2016-04-07T03:40:57.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12210 Milliseconds Vbucket 117 2016-04-07T03:40:57.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12240 Milliseconds Vbucket 117 2016-04-07T03:40:57.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12270 Milliseconds Vbucket 117 2016-04-07T03:40:57.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12300 Milliseconds Vbucket 117 2016-04-07T03:40:58.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12330 Milliseconds Vbucket 117 2016-04-07T03:40:58.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12360 Milliseconds Vbucket 117 2016-04-07T03:40:58.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12390 Milliseconds Vbucket 117 2016-04-07T03:40:58.097-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2580000 QueuedCount 50000 2016-04-07T03:40:58.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12420 Milliseconds Vbucket 117 2016-04-07T03:40:58.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12450 Milliseconds Vbucket 117 2016-04-07T03:40:58.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12480 Milliseconds Vbucket 117 2016-04-07T03:40:58.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12510 Milliseconds Vbucket 117 2016-04-07T03:40:58.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12540 Milliseconds Vbucket 117 2016-04-07T03:40:58.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12570 Milliseconds Vbucket 117 2016-04-07T03:40:58.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12600 Milliseconds Vbucket 117 2016-04-07T03:40:58.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12630 Milliseconds Vbucket 117 2016-04-07T03:40:58.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12660 Milliseconds Vbucket 117 2016-04-07T03:40:58.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12690 Milliseconds Vbucket 117 2016-04-07T03:40:58.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12720 Milliseconds Vbucket 117 2016-04-07T03:40:58.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12750 Milliseconds Vbucket 117 2016-04-07T03:40:58.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12780 Milliseconds Vbucket 117 2016-04-07T03:40:58.489-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2590000 QueuedCount 50000 2016-04-07T03:40:58.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12810 Milliseconds Vbucket 117 2016-04-07T03:40:58.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12840 Milliseconds Vbucket 117 2016-04-07T03:40:58.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12870 Milliseconds Vbucket 117 2016-04-07T03:40:58.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12900 Milliseconds Vbucket 117 2016-04-07T03:40:58.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12930 Milliseconds Vbucket 117 2016-04-07T03:40:58.648-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:40:58.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12960 Milliseconds Vbucket 117 2016-04-07T03:40:58.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12990 Milliseconds Vbucket 117 2016-04-07T03:40:58.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13020 Milliseconds Vbucket 117 2016-04-07T03:40:58.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13050 Milliseconds Vbucket 117 2016-04-07T03:40:58.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13080 Milliseconds Vbucket 117 2016-04-07T03:40:58.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13110 Milliseconds Vbucket 117 2016-04-07T03:40:58.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13140 Milliseconds Vbucket 117 2016-04-07T03:40:58.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13170 Milliseconds Vbucket 117 2016-04-07T03:40:58.881-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2600000 QueuedCount 50000 2016-04-07T03:40:58.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13200 Milliseconds Vbucket 117 2016-04-07T03:40:58.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13230 Milliseconds Vbucket 117 2016-04-07T03:40:58.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13260 Milliseconds Vbucket 117 2016-04-07T03:40:58.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13290 Milliseconds Vbucket 117 2016-04-07T03:40:59.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13320 Milliseconds Vbucket 117 2016-04-07T03:40:59.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13350 Milliseconds Vbucket 117 2016-04-07T03:40:59.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13380 Milliseconds Vbucket 117 2016-04-07T03:40:59.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13410 Milliseconds Vbucket 117 2016-04-07T03:40:59.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13440 Milliseconds Vbucket 117 2016-04-07T03:40:59.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13470 Milliseconds Vbucket 117 2016-04-07T03:40:59.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13500 Milliseconds Vbucket 117 2016-04-07T03:40:59.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13530 Milliseconds Vbucket 117 2016-04-07T03:40:59.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13560 Milliseconds Vbucket 117 2016-04-07T03:40:59.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13590 Milliseconds Vbucket 117 2016-04-07T03:40:59.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13620 Milliseconds Vbucket 117 2016-04-07T03:40:59.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13650 Milliseconds Vbucket 117 2016-04-07T03:40:59.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13680 Milliseconds Vbucket 117 2016-04-07T03:40:59.381-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2610000 QueuedCount 50000 2016-04-07T03:40:59.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13710 Milliseconds Vbucket 117 2016-04-07T03:40:59.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13740 Milliseconds Vbucket 117 2016-04-07T03:40:59.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13770 Milliseconds Vbucket 117 2016-04-07T03:40:59.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13800 Milliseconds Vbucket 117 2016-04-07T03:40:59.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13830 Milliseconds Vbucket 117 2016-04-07T03:40:59.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13860 Milliseconds Vbucket 117 2016-04-07T03:40:59.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13890 Milliseconds Vbucket 117 2016-04-07T03:40:59.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13920 Milliseconds Vbucket 117 2016-04-07T03:40:59.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13950 Milliseconds Vbucket 117 2016-04-07T03:40:59.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13980 Milliseconds Vbucket 117 2016-04-07T03:40:59.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14010 Milliseconds Vbucket 117 2016-04-07T03:40:59.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14040 Milliseconds Vbucket 117 2016-04-07T03:40:59.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14070 Milliseconds Vbucket 117 2016-04-07T03:40:59.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14100 Milliseconds Vbucket 117 2016-04-07T03:40:59.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14130 Milliseconds Vbucket 117 2016-04-07T03:40:59.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14160 Milliseconds Vbucket 117 2016-04-07T03:40:59.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14190 Milliseconds Vbucket 117 2016-04-07T03:40:59.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14220 Milliseconds Vbucket 117 2016-04-07T03:40:59.933-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2620000 QueuedCount 50000 2016-04-07T03:40:59.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14250 Milliseconds Vbucket 117 2016-04-07T03:40:59.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14280 Milliseconds Vbucket 117 2016-04-07T03:41:00.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14310 Milliseconds Vbucket 117 2016-04-07T03:41:00.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14340 Milliseconds Vbucket 117 2016-04-07T03:41:00.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14370 Milliseconds Vbucket 117 2016-04-07T03:41:00.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14400 Milliseconds Vbucket 117 2016-04-07T03:41:00.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14430 Milliseconds Vbucket 117 2016-04-07T03:41:00.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14460 Milliseconds Vbucket 117 2016-04-07T03:41:00.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14490 Milliseconds Vbucket 117 2016-04-07T03:41:00.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14520 Milliseconds Vbucket 117 2016-04-07T03:41:00.239-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2630000 QueuedCount 50000 2016-04-07T03:41:00.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14550 Milliseconds Vbucket 117 2016-04-07T03:41:00.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14580 Milliseconds Vbucket 117 2016-04-07T03:41:00.301-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:00.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14610 Milliseconds Vbucket 117 2016-04-07T03:41:00.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14640 Milliseconds Vbucket 117 2016-04-07T03:41:00.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14670 Milliseconds Vbucket 117 2016-04-07T03:41:00.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14700 Milliseconds Vbucket 117 2016-04-07T03:41:00.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14730 Milliseconds Vbucket 117 2016-04-07T03:41:00.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14760 Milliseconds Vbucket 117 2016-04-07T03:41:00.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14790 Milliseconds Vbucket 117 2016-04-07T03:41:00.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14820 Milliseconds Vbucket 117 2016-04-07T03:41:00.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14850 Milliseconds Vbucket 117 2016-04-07T03:41:00.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14880 Milliseconds Vbucket 117 2016-04-07T03:41:00.595-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2640000 QueuedCount 50000 2016-04-07T03:41:00.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14910 Milliseconds Vbucket 117 2016-04-07T03:41:00.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14940 Milliseconds Vbucket 117 2016-04-07T03:41:00.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14970 Milliseconds Vbucket 117 2016-04-07T03:41:00.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15000 Milliseconds Vbucket 117 2016-04-07T03:41:00.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15030 Milliseconds Vbucket 117 2016-04-07T03:41:00.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15060 Milliseconds Vbucket 117 2016-04-07T03:41:00.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15090 Milliseconds Vbucket 117 2016-04-07T03:41:00.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15120 Milliseconds Vbucket 117 2016-04-07T03:41:00.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15150 Milliseconds Vbucket 117 2016-04-07T03:41:00.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15180 Milliseconds Vbucket 117 2016-04-07T03:41:00.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15210 Milliseconds Vbucket 117 2016-04-07T03:41:00.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15240 Milliseconds Vbucket 117 2016-04-07T03:41:00.944-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2650000 QueuedCount 50000 2016-04-07T03:41:00.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15270 Milliseconds Vbucket 117 2016-04-07T03:41:00.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15300 Milliseconds Vbucket 117 2016-04-07T03:41:01.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15330 Milliseconds Vbucket 117 2016-04-07T03:41:01.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15360 Milliseconds Vbucket 117 2016-04-07T03:41:01.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15390 Milliseconds Vbucket 117 2016-04-07T03:41:01.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15420 Milliseconds Vbucket 117 2016-04-07T03:41:01.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15450 Milliseconds Vbucket 117 2016-04-07T03:41:01.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15480 Milliseconds Vbucket 117 2016-04-07T03:41:01.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15510 Milliseconds Vbucket 117 2016-04-07T03:41:01.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15540 Milliseconds Vbucket 117 2016-04-07T03:41:01.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15570 Milliseconds Vbucket 117 2016-04-07T03:41:01.264-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2660000 QueuedCount 47425 2016-04-07T03:41:01.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15600 Milliseconds Vbucket 117 2016-04-07T03:41:01.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15630 Milliseconds Vbucket 117 2016-04-07T03:41:01.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15660 Milliseconds Vbucket 117 2016-04-07T03:41:01.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15690 Milliseconds Vbucket 117 2016-04-07T03:41:01.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15720 Milliseconds Vbucket 117 2016-04-07T03:41:01.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15750 Milliseconds Vbucket 117 2016-04-07T03:41:01.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15780 Milliseconds Vbucket 117 2016-04-07T03:41:01.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15810 Milliseconds Vbucket 117 2016-04-07T03:41:01.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15840 Milliseconds Vbucket 117 2016-04-07T03:41:01.555-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2670000 QueuedCount 37425 2016-04-07T03:41:01.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15870 Milliseconds Vbucket 117 2016-04-07T03:41:01.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15900 Milliseconds Vbucket 117 2016-04-07T03:41:01.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15930 Milliseconds Vbucket 117 2016-04-07T03:41:01.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15960 Milliseconds Vbucket 117 2016-04-07T03:41:01.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15990 Milliseconds Vbucket 117 2016-04-07T03:41:01.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16020 Milliseconds Vbucket 117 2016-04-07T03:41:01.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16050 Milliseconds Vbucket 117 2016-04-07T03:41:01.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16080 Milliseconds Vbucket 117 2016-04-07T03:41:01.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16110 Milliseconds Vbucket 117 2016-04-07T03:41:01.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16140 Milliseconds Vbucket 117 2016-04-07T03:41:01.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16170 Milliseconds Vbucket 117 2016-04-07T03:41:01.872-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2680000 QueuedCount 27425 2016-04-07T03:41:01.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16200 Milliseconds Vbucket 117 2016-04-07T03:41:01.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16230 Milliseconds Vbucket 117 2016-04-07T03:41:01.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16260 Milliseconds Vbucket 117 2016-04-07T03:41:01.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16290 Milliseconds Vbucket 117 2016-04-07T03:41:02.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16320 Milliseconds Vbucket 117 2016-04-07T03:41:02.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16350 Milliseconds Vbucket 117 2016-04-07T03:41:02.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16380 Milliseconds Vbucket 117 2016-04-07T03:41:02.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16410 Milliseconds Vbucket 117 2016-04-07T03:41:02.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16440 Milliseconds Vbucket 117 2016-04-07T03:41:02.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16470 Milliseconds Vbucket 117 2016-04-07T03:41:02.175-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2690000 QueuedCount 17425 2016-04-07T03:41:02.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16500 Milliseconds Vbucket 117 2016-04-07T03:41:02.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16530 Milliseconds Vbucket 117 2016-04-07T03:41:02.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16560 Milliseconds Vbucket 117 2016-04-07T03:41:02.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16590 Milliseconds Vbucket 117 2016-04-07T03:41:02.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16620 Milliseconds Vbucket 117 2016-04-07T03:41:02.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16650 Milliseconds Vbucket 117 2016-04-07T03:41:02.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16680 Milliseconds Vbucket 117 2016-04-07T03:41:02.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16710 Milliseconds Vbucket 117 2016-04-07T03:41:02.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16740 Milliseconds Vbucket 117 2016-04-07T03:41:02.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16770 Milliseconds Vbucket 117 2016-04-07T03:41:02.464-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2700000 QueuedCount 7425 2016-04-07T03:41:02.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16800 Milliseconds Vbucket 117 2016-04-07T03:41:02.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16830 Milliseconds Vbucket 117 2016-04-07T03:41:02.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16860 Milliseconds Vbucket 117 2016-04-07T03:41:02.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16890 Milliseconds Vbucket 117 2016-04-07T03:41:02.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16920 Milliseconds Vbucket 117 2016-04-07T03:41:02.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16950 Milliseconds Vbucket 117 2016-04-07T03:41:02.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16980 Milliseconds Vbucket 117 2016-04-07T03:41:02.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17010 Milliseconds Vbucket 117 2016-04-07T03:41:02.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17040 Milliseconds Vbucket 117 2016-04-07T03:41:02.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17070 Milliseconds Vbucket 117 2016-04-07T03:41:02.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17100 Milliseconds Vbucket 117 2016-04-07T03:41:02.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17130 Milliseconds Vbucket 117 2016-04-07T03:41:02.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17160 Milliseconds Vbucket 117 2016-04-07T03:41:02.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17190 Milliseconds Vbucket 117 2016-04-07T03:41:02.912-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 13.429µs CommitTime 243.050369ms TotalFlushTime 1m25.521310467s TotalCommitTime 2.599216376s 2016-04-07T03:41:02.912-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 2707425, 2707425, 15 committed:true 2016-04-07T03:41:02.912-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 2707425, 2707425, 15 committed:true) SnapCreateDur 253.058681ms SnapOpenDur 55.747µs 2016-04-07T03:41:02.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17220 Milliseconds Vbucket 117 2016-04-07T03:41:02.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17250 Milliseconds Vbucket 117 2016-04-07T03:41:02.956-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:02.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17280 Milliseconds Vbucket 117 2016-04-07T03:41:03.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17310 Milliseconds Vbucket 117 2016-04-07T03:41:03.029-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2710000 QueuedCount 50000 2016-04-07T03:41:03.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17340 Milliseconds Vbucket 117 2016-04-07T03:41:03.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17370 Milliseconds Vbucket 117 2016-04-07T03:41:03.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17400 Milliseconds Vbucket 117 2016-04-07T03:41:03.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17430 Milliseconds Vbucket 117 2016-04-07T03:41:03.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17460 Milliseconds Vbucket 117 2016-04-07T03:41:03.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17490 Milliseconds Vbucket 117 2016-04-07T03:41:03.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17520 Milliseconds Vbucket 117 2016-04-07T03:41:03.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17550 Milliseconds Vbucket 117 2016-04-07T03:41:03.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17580 Milliseconds Vbucket 117 2016-04-07T03:41:03.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17610 Milliseconds Vbucket 117 2016-04-07T03:41:03.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17640 Milliseconds Vbucket 117 2016-04-07T03:41:03.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17670 Milliseconds Vbucket 117 2016-04-07T03:41:03.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17700 Milliseconds Vbucket 117 2016-04-07T03:41:03.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17730 Milliseconds Vbucket 117 2016-04-07T03:41:03.428-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2720000 QueuedCount 50000 2016-04-07T03:41:03.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17760 Milliseconds Vbucket 117 2016-04-07T03:41:03.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17790 Milliseconds Vbucket 117 2016-04-07T03:41:03.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17820 Milliseconds Vbucket 117 2016-04-07T03:41:03.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17850 Milliseconds Vbucket 117 2016-04-07T03:41:03.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17880 Milliseconds Vbucket 117 2016-04-07T03:41:03.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17910 Milliseconds Vbucket 117 2016-04-07T03:41:03.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17940 Milliseconds Vbucket 117 2016-04-07T03:41:03.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17970 Milliseconds Vbucket 117 2016-04-07T03:41:03.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18000 Milliseconds Vbucket 117 2016-04-07T03:41:03.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18030 Milliseconds Vbucket 117 2016-04-07T03:41:03.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18060 Milliseconds Vbucket 117 2016-04-07T03:41:03.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18090 Milliseconds Vbucket 117 2016-04-07T03:41:03.805-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2730000 QueuedCount 50000 2016-04-07T03:41:03.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18120 Milliseconds Vbucket 117 2016-04-07T03:41:03.818-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:03.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18150 Milliseconds Vbucket 117 2016-04-07T03:41:03.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18180 Milliseconds Vbucket 117 2016-04-07T03:41:03.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18210 Milliseconds Vbucket 117 2016-04-07T03:41:03.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18240 Milliseconds Vbucket 117 2016-04-07T03:41:03.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18270 Milliseconds Vbucket 117 2016-04-07T03:41:03.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18300 Milliseconds Vbucket 117 2016-04-07T03:41:04.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18330 Milliseconds Vbucket 117 2016-04-07T03:41:04.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18360 Milliseconds Vbucket 117 2016-04-07T03:41:04.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18390 Milliseconds Vbucket 117 2016-04-07T03:41:04.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18420 Milliseconds Vbucket 117 2016-04-07T03:41:04.118-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2740000 QueuedCount 50000 2016-04-07T03:41:04.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18450 Milliseconds Vbucket 117 2016-04-07T03:41:04.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18480 Milliseconds Vbucket 117 2016-04-07T03:41:04.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18510 Milliseconds Vbucket 117 2016-04-07T03:41:04.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18540 Milliseconds Vbucket 117 2016-04-07T03:41:04.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18570 Milliseconds Vbucket 117 2016-04-07T03:41:04.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18600 Milliseconds Vbucket 117 2016-04-07T03:41:04.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18630 Milliseconds Vbucket 117 2016-04-07T03:41:04.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18660 Milliseconds Vbucket 117 2016-04-07T03:41:04.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18690 Milliseconds Vbucket 117 2016-04-07T03:41:04.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18720 Milliseconds Vbucket 117 2016-04-07T03:41:04.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18750 Milliseconds Vbucket 117 2016-04-07T03:41:04.473-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2750000 QueuedCount 50000 2016-04-07T03:41:04.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18780 Milliseconds Vbucket 117 2016-04-07T03:41:04.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18810 Milliseconds Vbucket 117 2016-04-07T03:41:04.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18840 Milliseconds Vbucket 117 2016-04-07T03:41:04.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18870 Milliseconds Vbucket 117 2016-04-07T03:41:04.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18900 Milliseconds Vbucket 117 2016-04-07T03:41:04.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18930 Milliseconds Vbucket 117 2016-04-07T03:41:04.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18960 Milliseconds Vbucket 117 2016-04-07T03:41:04.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18990 Milliseconds Vbucket 117 2016-04-07T03:41:04.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19020 Milliseconds Vbucket 117 2016-04-07T03:41:04.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19050 Milliseconds Vbucket 117 2016-04-07T03:41:04.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19080 Milliseconds Vbucket 117 2016-04-07T03:41:04.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19110 Milliseconds Vbucket 117 2016-04-07T03:41:04.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19140 Milliseconds Vbucket 117 2016-04-07T03:41:04.849-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2760000 QueuedCount 50000 2016-04-07T03:41:04.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19170 Milliseconds Vbucket 117 2016-04-07T03:41:04.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19200 Milliseconds Vbucket 117 2016-04-07T03:41:04.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19230 Milliseconds Vbucket 117 2016-04-07T03:41:04.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19260 Milliseconds Vbucket 117 2016-04-07T03:41:04.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19290 Milliseconds Vbucket 117 2016-04-07T03:41:05.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19320 Milliseconds Vbucket 117 2016-04-07T03:41:05.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19350 Milliseconds Vbucket 117 2016-04-07T03:41:05.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19380 Milliseconds Vbucket 117 2016-04-07T03:41:05.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19410 Milliseconds Vbucket 117 2016-04-07T03:41:05.129-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:05.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19440 Milliseconds Vbucket 117 2016-04-07T03:41:05.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19470 Milliseconds Vbucket 117 2016-04-07T03:41:05.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19500 Milliseconds Vbucket 117 2016-04-07T03:41:05.2-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2770000 QueuedCount 50000 2016-04-07T03:41:05.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19530 Milliseconds Vbucket 117 2016-04-07T03:41:05.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19560 Milliseconds Vbucket 117 2016-04-07T03:41:05.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19590 Milliseconds Vbucket 117 2016-04-07T03:41:05.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19620 Milliseconds Vbucket 117 2016-04-07T03:41:05.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19650 Milliseconds Vbucket 117 2016-04-07T03:41:05.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19680 Milliseconds Vbucket 117 2016-04-07T03:41:05.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19710 Milliseconds Vbucket 117 2016-04-07T03:41:05.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19740 Milliseconds Vbucket 117 2016-04-07T03:41:05.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19770 Milliseconds Vbucket 117 2016-04-07T03:41:05.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19800 Milliseconds Vbucket 117 2016-04-07T03:41:05.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19830 Milliseconds Vbucket 117 2016-04-07T03:41:05.542-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2780000 QueuedCount 50000 2016-04-07T03:41:05.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19860 Milliseconds Vbucket 117 2016-04-07T03:41:05.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19890 Milliseconds Vbucket 117 2016-04-07T03:41:05.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19920 Milliseconds Vbucket 117 2016-04-07T03:41:05.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19950 Milliseconds Vbucket 117 2016-04-07T03:41:05.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19980 Milliseconds Vbucket 117 2016-04-07T03:41:05.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20010 Milliseconds Vbucket 117 2016-04-07T03:41:05.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20040 Milliseconds Vbucket 117 2016-04-07T03:41:05.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20070 Milliseconds Vbucket 117 2016-04-07T03:41:05.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20100 Milliseconds Vbucket 117 2016-04-07T03:41:05.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20130 Milliseconds Vbucket 117 2016-04-07T03:41:05.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20160 Milliseconds Vbucket 117 2016-04-07T03:41:05.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20190 Milliseconds Vbucket 117 2016-04-07T03:41:05.893-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2790000 QueuedCount 50000 2016-04-07T03:41:05.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20220 Milliseconds Vbucket 117 2016-04-07T03:41:05.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20250 Milliseconds Vbucket 117 2016-04-07T03:41:05.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20280 Milliseconds Vbucket 117 2016-04-07T03:41:06.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20310 Milliseconds Vbucket 117 2016-04-07T03:41:06.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20340 Milliseconds Vbucket 117 2016-04-07T03:41:06.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20370 Milliseconds Vbucket 117 2016-04-07T03:41:06.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20400 Milliseconds Vbucket 117 2016-04-07T03:41:06.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20430 Milliseconds Vbucket 117 2016-04-07T03:41:06.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20460 Milliseconds Vbucket 117 2016-04-07T03:41:06.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20490 Milliseconds Vbucket 117 2016-04-07T03:41:06.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20520 Milliseconds Vbucket 117 2016-04-07T03:41:06.233-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2800000 QueuedCount 50000 2016-04-07T03:41:06.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20550 Milliseconds Vbucket 117 2016-04-07T03:41:06.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20580 Milliseconds Vbucket 117 2016-04-07T03:41:06.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20610 Milliseconds Vbucket 117 2016-04-07T03:41:06.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20640 Milliseconds Vbucket 117 2016-04-07T03:41:06.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20670 Milliseconds Vbucket 117 2016-04-07T03:41:06.381-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:06.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20700 Milliseconds Vbucket 117 2016-04-07T03:41:06.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20730 Milliseconds Vbucket 117 2016-04-07T03:41:06.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20760 Milliseconds Vbucket 117 2016-04-07T03:41:06.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20790 Milliseconds Vbucket 117 2016-04-07T03:41:06.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20820 Milliseconds Vbucket 117 2016-04-07T03:41:06.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20850 Milliseconds Vbucket 117 2016-04-07T03:41:06.573-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2810000 QueuedCount 50000 2016-04-07T03:41:06.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20880 Milliseconds Vbucket 117 2016-04-07T03:41:06.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20910 Milliseconds Vbucket 117 2016-04-07T03:41:06.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20940 Milliseconds Vbucket 117 2016-04-07T03:41:06.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20970 Milliseconds Vbucket 117 2016-04-07T03:41:06.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21000 Milliseconds Vbucket 117 2016-04-07T03:41:06.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21030 Milliseconds Vbucket 117 2016-04-07T03:41:06.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21060 Milliseconds Vbucket 117 2016-04-07T03:41:06.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21090 Milliseconds Vbucket 117 2016-04-07T03:41:06.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21120 Milliseconds Vbucket 117 2016-04-07T03:41:06.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21150 Milliseconds Vbucket 117 2016-04-07T03:41:06.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21180 Milliseconds Vbucket 117 2016-04-07T03:41:06.899-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2820000 QueuedCount 50000 2016-04-07T03:41:06.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21210 Milliseconds Vbucket 117 2016-04-07T03:41:06.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21240 Milliseconds Vbucket 117 2016-04-07T03:41:06.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21270 Milliseconds Vbucket 117 2016-04-07T03:41:06.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21300 Milliseconds Vbucket 117 2016-04-07T03:41:07.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21330 Milliseconds Vbucket 117 2016-04-07T03:41:07.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21360 Milliseconds Vbucket 117 2016-04-07T03:41:07.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21390 Milliseconds Vbucket 117 2016-04-07T03:41:07.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21420 Milliseconds Vbucket 117 2016-04-07T03:41:07.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21450 Milliseconds Vbucket 117 2016-04-07T03:41:07.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21480 Milliseconds Vbucket 117 2016-04-07T03:41:07.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21510 Milliseconds Vbucket 117 2016-04-07T03:41:07.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21540 Milliseconds Vbucket 117 2016-04-07T03:41:07.246-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2830000 QueuedCount 50000 2016-04-07T03:41:07.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21570 Milliseconds Vbucket 117 2016-04-07T03:41:07.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21600 Milliseconds Vbucket 117 2016-04-07T03:41:07.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21630 Milliseconds Vbucket 117 2016-04-07T03:41:07.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21660 Milliseconds Vbucket 117 2016-04-07T03:41:07.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21690 Milliseconds Vbucket 117 2016-04-07T03:41:07.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21720 Milliseconds Vbucket 117 2016-04-07T03:41:07.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21750 Milliseconds Vbucket 117 2016-04-07T03:41:07.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21780 Milliseconds Vbucket 117 2016-04-07T03:41:07.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21810 Milliseconds Vbucket 117 2016-04-07T03:41:07.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21840 Milliseconds Vbucket 117 2016-04-07T03:41:07.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21870 Milliseconds Vbucket 117 2016-04-07T03:41:07.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21900 Milliseconds Vbucket 117 2016-04-07T03:41:07.609-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2840000 QueuedCount 50000 2016-04-07T03:41:07.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21930 Milliseconds Vbucket 117 2016-04-07T03:41:07.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21960 Milliseconds Vbucket 117 2016-04-07T03:41:07.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21990 Milliseconds Vbucket 117 2016-04-07T03:41:07.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22020 Milliseconds Vbucket 117 2016-04-07T03:41:07.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22050 Milliseconds Vbucket 117 2016-04-07T03:41:07.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22080 Milliseconds Vbucket 117 2016-04-07T03:41:07.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22110 Milliseconds Vbucket 117 2016-04-07T03:41:07.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22140 Milliseconds Vbucket 117 2016-04-07T03:41:07.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22170 Milliseconds Vbucket 117 2016-04-07T03:41:07.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22200 Milliseconds Vbucket 117 2016-04-07T03:41:07.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22230 Milliseconds Vbucket 117 2016-04-07T03:41:07.945-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2850000 QueuedCount 41588 2016-04-07T03:41:07.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22260 Milliseconds Vbucket 117 2016-04-07T03:41:07.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22290 Milliseconds Vbucket 117 2016-04-07T03:41:08.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22320 Milliseconds Vbucket 117 2016-04-07T03:41:08.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22350 Milliseconds Vbucket 117 2016-04-07T03:41:08.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22380 Milliseconds Vbucket 117 2016-04-07T03:41:08.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22410 Milliseconds Vbucket 117 2016-04-07T03:41:08.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22440 Milliseconds Vbucket 117 2016-04-07T03:41:08.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22470 Milliseconds Vbucket 117 2016-04-07T03:41:08.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22500 Milliseconds Vbucket 117 2016-04-07T03:41:08.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22530 Milliseconds Vbucket 117 2016-04-07T03:41:08.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22560 Milliseconds Vbucket 117 2016-04-07T03:41:08.259-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2860000 QueuedCount 31588 2016-04-07T03:41:08.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22590 Milliseconds Vbucket 117 2016-04-07T03:41:08.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22620 Milliseconds Vbucket 117 2016-04-07T03:41:08.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22650 Milliseconds Vbucket 117 2016-04-07T03:41:08.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22680 Milliseconds Vbucket 117 2016-04-07T03:41:08.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22710 Milliseconds Vbucket 117 2016-04-07T03:41:08.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22740 Milliseconds Vbucket 117 2016-04-07T03:41:08.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22770 Milliseconds Vbucket 117 2016-04-07T03:41:08.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22800 Milliseconds Vbucket 117 2016-04-07T03:41:08.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22830 Milliseconds Vbucket 117 2016-04-07T03:41:08.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22860 Milliseconds Vbucket 117 2016-04-07T03:41:08.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22890 Milliseconds Vbucket 117 2016-04-07T03:41:08.59-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2870000 QueuedCount 21588 2016-04-07T03:41:08.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22920 Milliseconds Vbucket 117 2016-04-07T03:41:08.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22950 Milliseconds Vbucket 117 2016-04-07T03:41:08.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22980 Milliseconds Vbucket 117 2016-04-07T03:41:08.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23010 Milliseconds Vbucket 117 2016-04-07T03:41:08.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23040 Milliseconds Vbucket 117 2016-04-07T03:41:08.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23070 Milliseconds Vbucket 117 2016-04-07T03:41:08.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23100 Milliseconds Vbucket 117 2016-04-07T03:41:08.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23130 Milliseconds Vbucket 117 2016-04-07T03:41:08.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23160 Milliseconds Vbucket 117 2016-04-07T03:41:08.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23190 Milliseconds Vbucket 117 2016-04-07T03:41:08.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23220 Milliseconds Vbucket 117 2016-04-07T03:41:08.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23250 Milliseconds Vbucket 117 2016-04-07T03:41:08.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23280 Milliseconds Vbucket 117 2016-04-07T03:41:09.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23310 Milliseconds Vbucket 117 2016-04-07T03:41:09.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23340 Milliseconds Vbucket 117 2016-04-07T03:41:09.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23370 Milliseconds Vbucket 117 2016-04-07T03:41:09.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23400 Milliseconds Vbucket 117 2016-04-07T03:41:09.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23430 Milliseconds Vbucket 117 2016-04-07T03:41:09.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23460 Milliseconds Vbucket 117 2016-04-07T03:41:09.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23490 Milliseconds Vbucket 117 2016-04-07T03:41:09.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23520 Milliseconds Vbucket 117 2016-04-07T03:41:09.243-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2880000 QueuedCount 11588 2016-04-07T03:41:09.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23550 Milliseconds Vbucket 117 2016-04-07T03:41:09.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23580 Milliseconds Vbucket 117 2016-04-07T03:41:09.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23610 Milliseconds Vbucket 117 2016-04-07T03:41:09.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23640 Milliseconds Vbucket 117 2016-04-07T03:41:09.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23670 Milliseconds Vbucket 117 2016-04-07T03:41:09.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23700 Milliseconds Vbucket 117 2016-04-07T03:41:09.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23730 Milliseconds Vbucket 117 2016-04-07T03:41:09.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23760 Milliseconds Vbucket 117 2016-04-07T03:41:09.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23790 Milliseconds Vbucket 117 2016-04-07T03:41:09.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23820 Milliseconds Vbucket 117 2016-04-07T03:41:09.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23850 Milliseconds Vbucket 117 2016-04-07T03:41:09.561-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2890000 QueuedCount 1588 2016-04-07T03:41:09.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23880 Milliseconds Vbucket 117 2016-04-07T03:41:09.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23910 Milliseconds Vbucket 117 2016-04-07T03:41:09.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23940 Milliseconds Vbucket 117 2016-04-07T03:41:09.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23970 Milliseconds Vbucket 117 2016-04-07T03:41:09.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24000 Milliseconds Vbucket 117 2016-04-07T03:41:09.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24030 Milliseconds Vbucket 117 2016-04-07T03:41:09.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24060 Milliseconds Vbucket 117 2016-04-07T03:41:09.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24090 Milliseconds Vbucket 117 2016-04-07T03:41:09.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24120 Milliseconds Vbucket 117 2016-04-07T03:41:09.842-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.764µs CommitTime 254.909486ms TotalFlushTime 1m31.854687862s TotalCommitTime 2.854125862s 2016-04-07T03:41:09.842-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 2891588, 2891588, 16 committed:true 2016-04-07T03:41:09.842-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 2891588, 2891588, 16 committed:true) SnapCreateDur 265.007329ms SnapOpenDur 52.672µs 2016-04-07T03:41:09.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24150 Milliseconds Vbucket 117 2016-04-07T03:41:09.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24180 Milliseconds Vbucket 117 2016-04-07T03:41:09.885-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:09.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24210 Milliseconds Vbucket 117 2016-04-07T03:41:09.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24240 Milliseconds Vbucket 117 2016-04-07T03:41:09.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24270 Milliseconds Vbucket 117 2016-04-07T03:41:09.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24300 Milliseconds Vbucket 117 2016-04-07T03:41:10.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24330 Milliseconds Vbucket 117 2016-04-07T03:41:10.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24360 Milliseconds Vbucket 117 2016-04-07T03:41:10.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24390 Milliseconds Vbucket 117 2016-04-07T03:41:10.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24420 Milliseconds Vbucket 117 2016-04-07T03:41:10.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24450 Milliseconds Vbucket 117 2016-04-07T03:41:10.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24480 Milliseconds Vbucket 117 2016-04-07T03:41:10.189-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2900000 QueuedCount 50000 2016-04-07T03:41:10.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24510 Milliseconds Vbucket 117 2016-04-07T03:41:10.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24540 Milliseconds Vbucket 117 2016-04-07T03:41:10.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24570 Milliseconds Vbucket 117 2016-04-07T03:41:10.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24600 Milliseconds Vbucket 117 2016-04-07T03:41:10.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24630 Milliseconds Vbucket 117 2016-04-07T03:41:10.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24660 Milliseconds Vbucket 117 2016-04-07T03:41:10.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24690 Milliseconds Vbucket 117 2016-04-07T03:41:10.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24720 Milliseconds Vbucket 117 2016-04-07T03:41:10.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24750 Milliseconds Vbucket 117 2016-04-07T03:41:10.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24780 Milliseconds Vbucket 117 2016-04-07T03:41:10.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24810 Milliseconds Vbucket 117 2016-04-07T03:41:10.519-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2910000 QueuedCount 50000 2016-04-07T03:41:10.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24840 Milliseconds Vbucket 117 2016-04-07T03:41:10.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24870 Milliseconds Vbucket 117 2016-04-07T03:41:10.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24900 Milliseconds Vbucket 117 2016-04-07T03:41:10.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24930 Milliseconds Vbucket 117 2016-04-07T03:41:10.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24960 Milliseconds Vbucket 117 2016-04-07T03:41:10.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24990 Milliseconds Vbucket 117 2016-04-07T03:41:10.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25020 Milliseconds Vbucket 117 2016-04-07T03:41:10.743-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:10.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25050 Milliseconds Vbucket 117 2016-04-07T03:41:10.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25080 Milliseconds Vbucket 117 2016-04-07T03:41:10.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25110 Milliseconds Vbucket 117 2016-04-07T03:41:10.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25140 Milliseconds Vbucket 117 2016-04-07T03:41:10.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25170 Milliseconds Vbucket 117 2016-04-07T03:41:10.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25200 Milliseconds Vbucket 117 2016-04-07T03:41:10.895-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2920000 QueuedCount 50000 2016-04-07T03:41:10.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25230 Milliseconds Vbucket 117 2016-04-07T03:41:10.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25260 Milliseconds Vbucket 117 2016-04-07T03:41:10.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25290 Milliseconds Vbucket 117 2016-04-07T03:41:11.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25320 Milliseconds Vbucket 117 2016-04-07T03:41:11.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25350 Milliseconds Vbucket 117 2016-04-07T03:41:11.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25380 Milliseconds Vbucket 117 2016-04-07T03:41:11.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25410 Milliseconds Vbucket 117 2016-04-07T03:41:11.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25440 Milliseconds Vbucket 117 2016-04-07T03:41:11.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25470 Milliseconds Vbucket 117 2016-04-07T03:41:11.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25500 Milliseconds Vbucket 117 2016-04-07T03:41:11.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25530 Milliseconds Vbucket 117 2016-04-07T03:41:11.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25560 Milliseconds Vbucket 117 2016-04-07T03:41:11.268-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2930000 QueuedCount 50000 2016-04-07T03:41:11.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25590 Milliseconds Vbucket 117 2016-04-07T03:41:11.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25620 Milliseconds Vbucket 117 2016-04-07T03:41:11.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25650 Milliseconds Vbucket 117 2016-04-07T03:41:11.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25680 Milliseconds Vbucket 117 2016-04-07T03:41:11.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25710 Milliseconds Vbucket 117 2016-04-07T03:41:11.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25740 Milliseconds Vbucket 117 2016-04-07T03:41:11.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25770 Milliseconds Vbucket 117 2016-04-07T03:41:11.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25800 Milliseconds Vbucket 117 2016-04-07T03:41:11.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25830 Milliseconds Vbucket 117 2016-04-07T03:41:11.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25860 Milliseconds Vbucket 117 2016-04-07T03:41:11.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25890 Milliseconds Vbucket 117 2016-04-07T03:41:11.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25920 Milliseconds Vbucket 117 2016-04-07T03:41:11.631-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2940000 QueuedCount 50000 2016-04-07T03:41:11.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25950 Milliseconds Vbucket 117 2016-04-07T03:41:11.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25980 Milliseconds Vbucket 117 2016-04-07T03:41:11.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26010 Milliseconds Vbucket 117 2016-04-07T03:41:11.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26040 Milliseconds Vbucket 117 2016-04-07T03:41:11.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26070 Milliseconds Vbucket 117 2016-04-07T03:41:11.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26100 Milliseconds Vbucket 117 2016-04-07T03:41:11.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26130 Milliseconds Vbucket 117 2016-04-07T03:41:11.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26160 Milliseconds Vbucket 117 2016-04-07T03:41:11.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26190 Milliseconds Vbucket 117 2016-04-07T03:41:11.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26220 Milliseconds Vbucket 117 2016-04-07T03:41:11.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26250 Milliseconds Vbucket 117 2016-04-07T03:41:11.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26280 Milliseconds Vbucket 117 2016-04-07T03:41:11.99-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2950000 QueuedCount 50000 2016-04-07T03:41:12.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26310 Milliseconds Vbucket 117 2016-04-07T03:41:12.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26340 Milliseconds Vbucket 117 2016-04-07T03:41:12.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26370 Milliseconds Vbucket 117 2016-04-07T03:41:12.067-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:12.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26400 Milliseconds Vbucket 117 2016-04-07T03:41:12.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26430 Milliseconds Vbucket 117 2016-04-07T03:41:12.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26460 Milliseconds Vbucket 117 2016-04-07T03:41:12.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26490 Milliseconds Vbucket 117 2016-04-07T03:41:12.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26520 Milliseconds Vbucket 117 2016-04-07T03:41:12.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26550 Milliseconds Vbucket 117 2016-04-07T03:41:12.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26580 Milliseconds Vbucket 117 2016-04-07T03:41:12.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26610 Milliseconds Vbucket 117 2016-04-07T03:41:12.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26640 Milliseconds Vbucket 117 2016-04-07T03:41:12.335-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2960000 QueuedCount 50000 2016-04-07T03:41:12.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26670 Milliseconds Vbucket 117 2016-04-07T03:41:12.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26700 Milliseconds Vbucket 117 2016-04-07T03:41:12.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26730 Milliseconds Vbucket 117 2016-04-07T03:41:12.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26760 Milliseconds Vbucket 117 2016-04-07T03:41:12.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26790 Milliseconds Vbucket 117 2016-04-07T03:41:12.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26820 Milliseconds Vbucket 117 2016-04-07T03:41:12.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26850 Milliseconds Vbucket 117 2016-04-07T03:41:12.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26880 Milliseconds Vbucket 117 2016-04-07T03:41:12.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26910 Milliseconds Vbucket 117 2016-04-07T03:41:12.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26940 Milliseconds Vbucket 117 2016-04-07T03:41:12.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26970 Milliseconds Vbucket 117 2016-04-07T03:41:12.683-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2970000 QueuedCount 50000 2016-04-07T03:41:12.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27000 Milliseconds Vbucket 117 2016-04-07T03:41:12.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27030 Milliseconds Vbucket 117 2016-04-07T03:41:12.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27060 Milliseconds Vbucket 117 2016-04-07T03:41:12.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27090 Milliseconds Vbucket 117 2016-04-07T03:41:12.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27120 Milliseconds Vbucket 117 2016-04-07T03:41:12.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27150 Milliseconds Vbucket 117 2016-04-07T03:41:12.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27180 Milliseconds Vbucket 117 2016-04-07T03:41:12.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27210 Milliseconds Vbucket 117 2016-04-07T03:41:12.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27240 Milliseconds Vbucket 117 2016-04-07T03:41:12.94-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8300164,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":6597768534,"bucket-1:myindex:avg_ts_items_count":165572,"bucket-1:myindex:build_progress":14,"bucket-1:myindex:data_size":921247601,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":2157772800,"bucket-1:myindex:flush_queue_size":50023,"bucket-1:myindex:frag_percent":57,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":308952000,"bucket-1:myindex:items_count":2891588,"bucket-1:myindex:num_commits":16,"bucket-1:myindex:num_compactions":1,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":2965745,"bucket-1:myindex:num_docs_pending":8672209,"bucket-1:myindex:num_docs_queued":8362046,"bucket-1:myindex:num_flush_queued":3027647,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":16,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":6929994013,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"16 2854125862 579390743716698682","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"2977625 15480899016 28672431523186788","bucket-1:myindex:timings/storage_handle_open":"16 396579 10191174673","bucket-1:myindex:timings/storage_info":"65 459236 6774823278","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"16 20595020 265155608653584","bucket-1:myindex:timings/storage_meta_set":"16 5696529 2177195618837","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"5955249 77405818327 2476029243636157571","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":11327789,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":264,"memory_quota":31457280000,"memory_used":2684461056,"needs_restart":false,"num_connections":0,"timings/stats_response":"1202 79250435 10156655498461"} 2016-04-07T03:41:12.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27270 Milliseconds Vbucket 117 2016-04-07T03:41:12.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27300 Milliseconds Vbucket 117 2016-04-07T03:41:13.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27330 Milliseconds Vbucket 117 2016-04-07T03:41:13.035-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2980000 QueuedCount 50000 2016-04-07T03:41:13.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27360 Milliseconds Vbucket 117 2016-04-07T03:41:13.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27390 Milliseconds Vbucket 117 2016-04-07T03:41:13.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27420 Milliseconds Vbucket 117 2016-04-07T03:41:13.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27450 Milliseconds Vbucket 117 2016-04-07T03:41:13.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27480 Milliseconds Vbucket 117 2016-04-07T03:41:13.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27510 Milliseconds Vbucket 117 2016-04-07T03:41:13.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27540 Milliseconds Vbucket 117 2016-04-07T03:41:13.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27570 Milliseconds Vbucket 117 2016-04-07T03:41:13.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27600 Milliseconds Vbucket 117 2016-04-07T03:41:13.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27630 Milliseconds Vbucket 117 2016-04-07T03:41:13.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27660 Milliseconds Vbucket 117 2016-04-07T03:41:13.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27690 Milliseconds Vbucket 117 2016-04-07T03:41:13.386-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 2990000 QueuedCount 49786 2016-04-07T03:41:13.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27720 Milliseconds Vbucket 117 2016-04-07T03:41:13.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27750 Milliseconds Vbucket 117 2016-04-07T03:41:13.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27780 Milliseconds Vbucket 117 2016-04-07T03:41:13.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27810 Milliseconds Vbucket 117 2016-04-07T03:41:13.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27840 Milliseconds Vbucket 117 2016-04-07T03:41:13.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27870 Milliseconds Vbucket 117 2016-04-07T03:41:13.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27900 Milliseconds Vbucket 117 2016-04-07T03:41:13.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27930 Milliseconds Vbucket 117 2016-04-07T03:41:13.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27960 Milliseconds Vbucket 117 2016-04-07T03:41:13.659-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3000000 QueuedCount 39786 2016-04-07T03:41:13.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27990 Milliseconds Vbucket 117 2016-04-07T03:41:13.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28020 Milliseconds Vbucket 117 2016-04-07T03:41:13.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28050 Milliseconds Vbucket 117 2016-04-07T03:41:13.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28080 Milliseconds Vbucket 117 2016-04-07T03:41:13.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28110 Milliseconds Vbucket 117 2016-04-07T03:41:13.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28140 Milliseconds Vbucket 117 2016-04-07T03:41:13.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28170 Milliseconds Vbucket 117 2016-04-07T03:41:13.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28200 Milliseconds Vbucket 117 2016-04-07T03:41:13.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28230 Milliseconds Vbucket 117 2016-04-07T03:41:13.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28260 Milliseconds Vbucket 117 2016-04-07T03:41:13.966-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3010000 QueuedCount 29786 2016-04-07T03:41:13.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28290 Milliseconds Vbucket 117 2016-04-07T03:41:14.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28320 Milliseconds Vbucket 117 2016-04-07T03:41:14.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28350 Milliseconds Vbucket 117 2016-04-07T03:41:14.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28380 Milliseconds Vbucket 117 2016-04-07T03:41:14.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28410 Milliseconds Vbucket 117 2016-04-07T03:41:14.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28440 Milliseconds Vbucket 117 2016-04-07T03:41:14.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28470 Milliseconds Vbucket 117 2016-04-07T03:41:14.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28500 Milliseconds Vbucket 117 2016-04-07T03:41:14.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28530 Milliseconds Vbucket 117 2016-04-07T03:41:14.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28560 Milliseconds Vbucket 117 2016-04-07T03:41:14.284-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3020000 QueuedCount 19786 2016-04-07T03:41:14.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28590 Milliseconds Vbucket 117 2016-04-07T03:41:14.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28620 Milliseconds Vbucket 117 2016-04-07T03:41:14.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28650 Milliseconds Vbucket 117 2016-04-07T03:41:14.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28680 Milliseconds Vbucket 117 2016-04-07T03:41:14.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28710 Milliseconds Vbucket 117 2016-04-07T03:41:14.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28740 Milliseconds Vbucket 117 2016-04-07T03:41:14.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28770 Milliseconds Vbucket 117 2016-04-07T03:41:14.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28800 Milliseconds Vbucket 117 2016-04-07T03:41:14.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28830 Milliseconds Vbucket 117 2016-04-07T03:41:14.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28860 Milliseconds Vbucket 117 2016-04-07T03:41:14.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28890 Milliseconds Vbucket 117 2016-04-07T03:41:14.591-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3030000 QueuedCount 9786 2016-04-07T03:41:14.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28920 Milliseconds Vbucket 117 2016-04-07T03:41:14.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28950 Milliseconds Vbucket 117 2016-04-07T03:41:14.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28980 Milliseconds Vbucket 117 2016-04-07T03:41:14.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29010 Milliseconds Vbucket 117 2016-04-07T03:41:14.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29040 Milliseconds Vbucket 117 2016-04-07T03:41:14.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29070 Milliseconds Vbucket 117 2016-04-07T03:41:14.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29100 Milliseconds Vbucket 117 2016-04-07T03:41:14.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29130 Milliseconds Vbucket 117 2016-04-07T03:41:14.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29160 Milliseconds Vbucket 117 2016-04-07T03:41:14.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29190 Milliseconds Vbucket 117 2016-04-07T03:41:14.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29220 Milliseconds Vbucket 117 2016-04-07T03:41:14.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29250 Milliseconds Vbucket 117 2016-04-07T03:41:14.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29280 Milliseconds Vbucket 117 2016-04-07T03:41:15.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29310 Milliseconds Vbucket 117 2016-04-07T03:41:15.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29340 Milliseconds Vbucket 117 2016-04-07T03:41:15.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29370 Milliseconds Vbucket 117 2016-04-07T03:41:15.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29400 Milliseconds Vbucket 117 2016-04-07T03:41:15.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29430 Milliseconds Vbucket 117 2016-04-07T03:41:15.136-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.739µs CommitTime 225.992889ms TotalFlushTime 1m36.664720547s TotalCommitTime 3.080118751s 2016-04-07T03:41:15.136-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 3039786, 3039786, 17 committed:true 2016-04-07T03:41:15.136-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 3039786, 3039786, 17 committed:true) SnapCreateDur 235.747757ms SnapOpenDur 54.883µs 2016-04-07T03:41:15.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29460 Milliseconds Vbucket 117 2016-04-07T03:41:15.162-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3040000 QueuedCount 9712 2016-04-07T03:41:15.18-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:15.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29490 Milliseconds Vbucket 117 2016-04-07T03:41:15.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29520 Milliseconds Vbucket 117 2016-04-07T03:41:15.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29550 Milliseconds Vbucket 117 2016-04-07T03:41:15.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29580 Milliseconds Vbucket 117 2016-04-07T03:41:15.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29610 Milliseconds Vbucket 117 2016-04-07T03:41:15.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29640 Milliseconds Vbucket 117 2016-04-07T03:41:15.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29670 Milliseconds Vbucket 117 2016-04-07T03:41:15.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29700 Milliseconds Vbucket 117 2016-04-07T03:41:15.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29730 Milliseconds Vbucket 117 2016-04-07T03:41:15.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29760 Milliseconds Vbucket 117 2016-04-07T03:41:15.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29790 Milliseconds Vbucket 117 2016-04-07T03:41:15.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29820 Milliseconds Vbucket 117 2016-04-07T03:41:15.524-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3050000 QueuedCount 50000 2016-04-07T03:41:15.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29850 Milliseconds Vbucket 117 2016-04-07T03:41:15.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29880 Milliseconds Vbucket 117 2016-04-07T03:41:15.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29910 Milliseconds Vbucket 117 2016-04-07T03:41:15.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29940 Milliseconds Vbucket 117 2016-04-07T03:41:15.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29970 Milliseconds Vbucket 117 2016-04-07T03:41:15.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30000 Milliseconds Vbucket 117 2016-04-07T03:41:15.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30030 Milliseconds Vbucket 117 2016-04-07T03:41:15.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30060 Milliseconds Vbucket 117 2016-04-07T03:41:15.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30090 Milliseconds Vbucket 117 2016-04-07T03:41:15.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30120 Milliseconds Vbucket 117 2016-04-07T03:41:15.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30150 Milliseconds Vbucket 117 2016-04-07T03:41:15.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30180 Milliseconds Vbucket 117 2016-04-07T03:41:15.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30210 Milliseconds Vbucket 117 2016-04-07T03:41:15.908-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3060000 QueuedCount 50000 2016-04-07T03:41:15.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30240 Milliseconds Vbucket 117 2016-04-07T03:41:15.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30270 Milliseconds Vbucket 117 2016-04-07T03:41:15.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30300 Milliseconds Vbucket 117 2016-04-07T03:41:16.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30330 Milliseconds Vbucket 117 2016-04-07T03:41:16.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30360 Milliseconds Vbucket 117 2016-04-07T03:41:16.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30390 Milliseconds Vbucket 117 2016-04-07T03:41:16.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30420 Milliseconds Vbucket 117 2016-04-07T03:41:16.124-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:16.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30450 Milliseconds Vbucket 117 2016-04-07T03:41:16.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30480 Milliseconds Vbucket 117 2016-04-07T03:41:16.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30510 Milliseconds Vbucket 117 2016-04-07T03:41:16.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30540 Milliseconds Vbucket 117 2016-04-07T03:41:16.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30570 Milliseconds Vbucket 117 2016-04-07T03:41:16.283-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3070000 QueuedCount 50000 2016-04-07T03:41:16.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30600 Milliseconds Vbucket 117 2016-04-07T03:41:16.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30630 Milliseconds Vbucket 117 2016-04-07T03:41:16.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30660 Milliseconds Vbucket 117 2016-04-07T03:41:16.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30690 Milliseconds Vbucket 117 2016-04-07T03:41:16.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30720 Milliseconds Vbucket 117 2016-04-07T03:41:16.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30750 Milliseconds Vbucket 117 2016-04-07T03:41:16.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30780 Milliseconds Vbucket 117 2016-04-07T03:41:16.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30810 Milliseconds Vbucket 117 2016-04-07T03:41:16.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30840 Milliseconds Vbucket 117 2016-04-07T03:41:16.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30870 Milliseconds Vbucket 117 2016-04-07T03:41:16.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30900 Milliseconds Vbucket 117 2016-04-07T03:41:16.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30930 Milliseconds Vbucket 117 2016-04-07T03:41:16.643-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3080000 QueuedCount 50000 2016-04-07T03:41:16.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30960 Milliseconds Vbucket 117 2016-04-07T03:41:16.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30990 Milliseconds Vbucket 117 2016-04-07T03:41:16.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31020 Milliseconds Vbucket 117 2016-04-07T03:41:16.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31050 Milliseconds Vbucket 117 2016-04-07T03:41:16.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31080 Milliseconds Vbucket 117 2016-04-07T03:41:16.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31110 Milliseconds Vbucket 117 2016-04-07T03:41:16.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31140 Milliseconds Vbucket 117 2016-04-07T03:41:16.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31170 Milliseconds Vbucket 117 2016-04-07T03:41:16.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31200 Milliseconds Vbucket 117 2016-04-07T03:41:16.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31230 Milliseconds Vbucket 117 2016-04-07T03:41:16.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31260 Milliseconds Vbucket 117 2016-04-07T03:41:16.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31290 Milliseconds Vbucket 117 2016-04-07T03:41:17.006-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3090000 QueuedCount 50000 2016-04-07T03:41:17.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31320 Milliseconds Vbucket 117 2016-04-07T03:41:17.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31350 Milliseconds Vbucket 117 2016-04-07T03:41:17.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31380 Milliseconds Vbucket 117 2016-04-07T03:41:17.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31410 Milliseconds Vbucket 117 2016-04-07T03:41:17.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31440 Milliseconds Vbucket 117 2016-04-07T03:41:17.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31470 Milliseconds Vbucket 117 2016-04-07T03:41:17.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31500 Milliseconds Vbucket 117 2016-04-07T03:41:17.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31530 Milliseconds Vbucket 117 2016-04-07T03:41:17.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31560 Milliseconds Vbucket 117 2016-04-07T03:41:17.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31590 Milliseconds Vbucket 117 2016-04-07T03:41:17.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31620 Milliseconds Vbucket 117 2016-04-07T03:41:17.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31650 Milliseconds Vbucket 117 2016-04-07T03:41:17.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31680 Milliseconds Vbucket 117 2016-04-07T03:41:17.375-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3100000 QueuedCount 50000 2016-04-07T03:41:17.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31710 Milliseconds Vbucket 117 2016-04-07T03:41:17.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31740 Milliseconds Vbucket 117 2016-04-07T03:41:17.451-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:17.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31770 Milliseconds Vbucket 117 2016-04-07T03:41:17.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31800 Milliseconds Vbucket 117 2016-04-07T03:41:17.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31830 Milliseconds Vbucket 117 2016-04-07T03:41:17.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31860 Milliseconds Vbucket 117 2016-04-07T03:41:17.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31890 Milliseconds Vbucket 117 2016-04-07T03:41:17.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31920 Milliseconds Vbucket 117 2016-04-07T03:41:17.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31950 Milliseconds Vbucket 117 2016-04-07T03:41:17.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31980 Milliseconds Vbucket 117 2016-04-07T03:41:17.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32010 Milliseconds Vbucket 117 2016-04-07T03:41:17.73-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3110000 QueuedCount 50000 2016-04-07T03:41:17.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32040 Milliseconds Vbucket 117 2016-04-07T03:41:17.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32070 Milliseconds Vbucket 117 2016-04-07T03:41:17.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32100 Milliseconds Vbucket 117 2016-04-07T03:41:17.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32130 Milliseconds Vbucket 117 2016-04-07T03:41:17.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32160 Milliseconds Vbucket 117 2016-04-07T03:41:17.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32190 Milliseconds Vbucket 117 2016-04-07T03:41:17.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32220 Milliseconds Vbucket 117 2016-04-07T03:41:17.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32250 Milliseconds Vbucket 117 2016-04-07T03:41:17.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32280 Milliseconds Vbucket 117 2016-04-07T03:41:18.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32310 Milliseconds Vbucket 117 2016-04-07T03:41:18.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32340 Milliseconds Vbucket 117 2016-04-07T03:41:18.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32370 Milliseconds Vbucket 117 2016-04-07T03:41:18.088-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3120000 QueuedCount 50000 2016-04-07T03:41:18.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32400 Milliseconds Vbucket 117 2016-04-07T03:41:18.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32430 Milliseconds Vbucket 117 2016-04-07T03:41:18.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32460 Milliseconds Vbucket 117 2016-04-07T03:41:18.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32490 Milliseconds Vbucket 117 2016-04-07T03:41:18.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32520 Milliseconds Vbucket 117 2016-04-07T03:41:18.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32550 Milliseconds Vbucket 117 2016-04-07T03:41:18.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32580 Milliseconds Vbucket 117 2016-04-07T03:41:18.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32610 Milliseconds Vbucket 117 2016-04-07T03:41:18.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32640 Milliseconds Vbucket 117 2016-04-07T03:41:18.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32670 Milliseconds Vbucket 117 2016-04-07T03:41:18.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32700 Milliseconds Vbucket 117 2016-04-07T03:41:18.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32730 Milliseconds Vbucket 117 2016-04-07T03:41:18.44-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3130000 QueuedCount 50000 2016-04-07T03:41:18.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32760 Milliseconds Vbucket 117 2016-04-07T03:41:18.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32790 Milliseconds Vbucket 117 2016-04-07T03:41:18.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32820 Milliseconds Vbucket 117 2016-04-07T03:41:18.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32850 Milliseconds Vbucket 117 2016-04-07T03:41:18.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32880 Milliseconds Vbucket 117 2016-04-07T03:41:18.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32910 Milliseconds Vbucket 117 2016-04-07T03:41:18.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32940 Milliseconds Vbucket 117 2016-04-07T03:41:18.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32970 Milliseconds Vbucket 117 2016-04-07T03:41:18.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33000 Milliseconds Vbucket 117 2016-04-07T03:41:18.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33030 Milliseconds Vbucket 117 2016-04-07T03:41:18.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33060 Milliseconds Vbucket 117 2016-04-07T03:41:18.766-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3140000 QueuedCount 50000 2016-04-07T03:41:18.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33090 Milliseconds Vbucket 117 2016-04-07T03:41:18.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33120 Milliseconds Vbucket 117 2016-04-07T03:41:18.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33150 Milliseconds Vbucket 117 2016-04-07T03:41:18.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33180 Milliseconds Vbucket 117 2016-04-07T03:41:18.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33210 Milliseconds Vbucket 117 2016-04-07T03:41:18.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33240 Milliseconds Vbucket 117 2016-04-07T03:41:18.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33270 Milliseconds Vbucket 117 2016-04-07T03:41:18.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33300 Milliseconds Vbucket 117 2016-04-07T03:41:19.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33330 Milliseconds Vbucket 117 2016-04-07T03:41:19.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33360 Milliseconds Vbucket 117 2016-04-07T03:41:19.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33390 Milliseconds Vbucket 117 2016-04-07T03:41:19.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33420 Milliseconds Vbucket 117 2016-04-07T03:41:19.128-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3150000 QueuedCount 40300 2016-04-07T03:41:19.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33450 Milliseconds Vbucket 117 2016-04-07T03:41:19.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33480 Milliseconds Vbucket 117 2016-04-07T03:41:19.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33510 Milliseconds Vbucket 117 2016-04-07T03:41:19.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33540 Milliseconds Vbucket 117 2016-04-07T03:41:19.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33570 Milliseconds Vbucket 117 2016-04-07T03:41:19.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33600 Milliseconds Vbucket 117 2016-04-07T03:41:19.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33630 Milliseconds Vbucket 117 2016-04-07T03:41:19.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33660 Milliseconds Vbucket 117 2016-04-07T03:41:19.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33690 Milliseconds Vbucket 117 2016-04-07T03:41:19.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33720 Milliseconds Vbucket 117 2016-04-07T03:41:19.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33750 Milliseconds Vbucket 117 2016-04-07T03:41:19.466-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3160000 QueuedCount 30300 2016-04-07T03:41:19.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33780 Milliseconds Vbucket 117 2016-04-07T03:41:19.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33810 Milliseconds Vbucket 117 2016-04-07T03:41:19.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33840 Milliseconds Vbucket 117 2016-04-07T03:41:19.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33870 Milliseconds Vbucket 117 2016-04-07T03:41:19.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33900 Milliseconds Vbucket 117 2016-04-07T03:41:19.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33930 Milliseconds Vbucket 117 2016-04-07T03:41:19.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33960 Milliseconds Vbucket 117 2016-04-07T03:41:19.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33990 Milliseconds Vbucket 117 2016-04-07T03:41:19.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34020 Milliseconds Vbucket 117 2016-04-07T03:41:19.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34050 Milliseconds Vbucket 117 2016-04-07T03:41:19.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34080 Milliseconds Vbucket 117 2016-04-07T03:41:19.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34110 Milliseconds Vbucket 117 2016-04-07T03:41:19.804-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3170000 QueuedCount 20300 2016-04-07T03:41:19.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34140 Milliseconds Vbucket 117 2016-04-07T03:41:19.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34170 Milliseconds Vbucket 117 2016-04-07T03:41:19.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34200 Milliseconds Vbucket 117 2016-04-07T03:41:19.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34230 Milliseconds Vbucket 117 2016-04-07T03:41:19.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34260 Milliseconds Vbucket 117 2016-04-07T03:41:19.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34290 Milliseconds Vbucket 117 2016-04-07T03:41:20.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34320 Milliseconds Vbucket 117 2016-04-07T03:41:20.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34350 Milliseconds Vbucket 117 2016-04-07T03:41:20.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34380 Milliseconds Vbucket 117 2016-04-07T03:41:20.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34410 Milliseconds Vbucket 117 2016-04-07T03:41:20.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34440 Milliseconds Vbucket 117 2016-04-07T03:41:20.14-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3180000 QueuedCount 10300 2016-04-07T03:41:20.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34470 Milliseconds Vbucket 117 2016-04-07T03:41:20.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34500 Milliseconds Vbucket 117 2016-04-07T03:41:20.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34530 Milliseconds Vbucket 117 2016-04-07T03:41:20.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34560 Milliseconds Vbucket 117 2016-04-07T03:41:20.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34590 Milliseconds Vbucket 117 2016-04-07T03:41:20.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34620 Milliseconds Vbucket 117 2016-04-07T03:41:20.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34650 Milliseconds Vbucket 117 2016-04-07T03:41:20.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34680 Milliseconds Vbucket 117 2016-04-07T03:41:20.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34710 Milliseconds Vbucket 117 2016-04-07T03:41:20.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34740 Milliseconds Vbucket 117 2016-04-07T03:41:20.46-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3190000 QueuedCount 300 2016-04-07T03:41:20.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34770 Milliseconds Vbucket 117 2016-04-07T03:41:20.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34800 Milliseconds Vbucket 117 2016-04-07T03:41:20.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34830 Milliseconds Vbucket 117 2016-04-07T03:41:20.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34860 Milliseconds Vbucket 117 2016-04-07T03:41:20.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34890 Milliseconds Vbucket 117 2016-04-07T03:41:20.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34920 Milliseconds Vbucket 117 2016-04-07T03:41:20.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34950 Milliseconds Vbucket 117 2016-04-07T03:41:20.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34980 Milliseconds Vbucket 117 2016-04-07T03:41:20.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35010 Milliseconds Vbucket 117 2016-04-07T03:41:20.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35040 Milliseconds Vbucket 117 2016-04-07T03:41:20.748-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 10.538µs CommitTime 269.884739ms TotalFlushTime 1m41.740525707s TotalCommitTime 3.35000349s 2016-04-07T03:41:20.748-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 3190300, 3190300, 18 committed:true 2016-04-07T03:41:20.748-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 3190300, 3190300, 18 committed:true) SnapCreateDur 280.031572ms SnapOpenDur 50.691µs 2016-04-07T03:41:20.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35070 Milliseconds Vbucket 117 2016-04-07T03:41:20.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35100 Milliseconds Vbucket 117 2016-04-07T03:41:20.795-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:20.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35130 Milliseconds Vbucket 117 2016-04-07T03:41:20.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35160 Milliseconds Vbucket 117 2016-04-07T03:41:20.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35190 Milliseconds Vbucket 117 2016-04-07T03:41:20.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35220 Milliseconds Vbucket 117 2016-04-07T03:41:20.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35250 Milliseconds Vbucket 117 2016-04-07T03:41:20.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35280 Milliseconds Vbucket 117 2016-04-07T03:41:21.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35310 Milliseconds Vbucket 117 2016-04-07T03:41:21.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35340 Milliseconds Vbucket 117 2016-04-07T03:41:21.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35370 Milliseconds Vbucket 117 2016-04-07T03:41:21.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35400 Milliseconds Vbucket 117 2016-04-07T03:41:21.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35430 Milliseconds Vbucket 117 2016-04-07T03:41:21.126-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3200000 QueuedCount 50000 2016-04-07T03:41:21.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35460 Milliseconds Vbucket 117 2016-04-07T03:41:21.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35490 Milliseconds Vbucket 117 2016-04-07T03:41:21.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35520 Milliseconds Vbucket 117 2016-04-07T03:41:21.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35550 Milliseconds Vbucket 117 2016-04-07T03:41:21.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35580 Milliseconds Vbucket 117 2016-04-07T03:41:21.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35610 Milliseconds Vbucket 117 2016-04-07T03:41:21.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35640 Milliseconds Vbucket 117 2016-04-07T03:41:21.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35670 Milliseconds Vbucket 117 2016-04-07T03:41:21.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35700 Milliseconds Vbucket 117 2016-04-07T03:41:21.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35730 Milliseconds Vbucket 117 2016-04-07T03:41:21.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35760 Milliseconds Vbucket 117 2016-04-07T03:41:21.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35790 Milliseconds Vbucket 117 2016-04-07T03:41:21.492-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3210000 QueuedCount 50000 2016-04-07T03:41:21.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35820 Milliseconds Vbucket 117 2016-04-07T03:41:21.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35850 Milliseconds Vbucket 117 2016-04-07T03:41:21.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35880 Milliseconds Vbucket 117 2016-04-07T03:41:21.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35910 Milliseconds Vbucket 117 2016-04-07T03:41:21.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35940 Milliseconds Vbucket 117 2016-04-07T03:41:21.635-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:21.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35970 Milliseconds Vbucket 117 2016-04-07T03:41:21.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36000 Milliseconds Vbucket 117 2016-04-07T03:41:21.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36030 Milliseconds Vbucket 117 2016-04-07T03:41:21.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36060 Milliseconds Vbucket 117 2016-04-07T03:41:21.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36090 Milliseconds Vbucket 117 2016-04-07T03:41:21.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36120 Milliseconds Vbucket 117 2016-04-07T03:41:21.826-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3220000 QueuedCount 50000 2016-04-07T03:41:21.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36150 Milliseconds Vbucket 117 2016-04-07T03:41:21.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36180 Milliseconds Vbucket 117 2016-04-07T03:41:21.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36210 Milliseconds Vbucket 117 2016-04-07T03:41:21.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36240 Milliseconds Vbucket 117 2016-04-07T03:41:21.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36270 Milliseconds Vbucket 117 2016-04-07T03:41:21.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36300 Milliseconds Vbucket 117 2016-04-07T03:41:22.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36330 Milliseconds Vbucket 117 2016-04-07T03:41:22.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36360 Milliseconds Vbucket 117 2016-04-07T03:41:22.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36390 Milliseconds Vbucket 117 2016-04-07T03:41:22.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36420 Milliseconds Vbucket 117 2016-04-07T03:41:22.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36450 Milliseconds Vbucket 117 2016-04-07T03:41:22.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36480 Milliseconds Vbucket 117 2016-04-07T03:41:22.19-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3230000 QueuedCount 50000 2016-04-07T03:41:22.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36510 Milliseconds Vbucket 117 2016-04-07T03:41:22.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36540 Milliseconds Vbucket 117 2016-04-07T03:41:22.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36570 Milliseconds Vbucket 117 2016-04-07T03:41:22.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36600 Milliseconds Vbucket 117 2016-04-07T03:41:22.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36630 Milliseconds Vbucket 117 2016-04-07T03:41:22.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36660 Milliseconds Vbucket 117 2016-04-07T03:41:22.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36690 Milliseconds Vbucket 117 2016-04-07T03:41:22.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36720 Milliseconds Vbucket 117 2016-04-07T03:41:22.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36750 Milliseconds Vbucket 117 2016-04-07T03:41:22.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36780 Milliseconds Vbucket 117 2016-04-07T03:41:22.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36810 Milliseconds Vbucket 117 2016-04-07T03:41:22.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36840 Milliseconds Vbucket 117 2016-04-07T03:41:22.553-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3240000 QueuedCount 50000 2016-04-07T03:41:22.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36870 Milliseconds Vbucket 117 2016-04-07T03:41:22.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36900 Milliseconds Vbucket 117 2016-04-07T03:41:22.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36930 Milliseconds Vbucket 117 2016-04-07T03:41:22.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36960 Milliseconds Vbucket 117 2016-04-07T03:41:22.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36990 Milliseconds Vbucket 117 2016-04-07T03:41:22.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37020 Milliseconds Vbucket 117 2016-04-07T03:41:22.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37050 Milliseconds Vbucket 117 2016-04-07T03:41:22.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37080 Milliseconds Vbucket 117 2016-04-07T03:41:22.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37110 Milliseconds Vbucket 117 2016-04-07T03:41:22.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37140 Milliseconds Vbucket 117 2016-04-07T03:41:22.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37170 Milliseconds Vbucket 117 2016-04-07T03:41:22.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37200 Milliseconds Vbucket 117 2016-04-07T03:41:22.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37230 Milliseconds Vbucket 117 2016-04-07T03:41:22.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37260 Milliseconds Vbucket 117 2016-04-07T03:41:22.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37290 Milliseconds Vbucket 117 2016-04-07T03:41:23.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37320 Milliseconds Vbucket 117 2016-04-07T03:41:23.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37350 Milliseconds Vbucket 117 2016-04-07T03:41:23.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3250000 QueuedCount 50000 2016-04-07T03:41:23.065-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:23.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37380 Milliseconds Vbucket 117 2016-04-07T03:41:23.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37410 Milliseconds Vbucket 117 2016-04-07T03:41:23.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37440 Milliseconds Vbucket 117 2016-04-07T03:41:23.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37470 Milliseconds Vbucket 117 2016-04-07T03:41:23.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37500 Milliseconds Vbucket 117 2016-04-07T03:41:23.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37530 Milliseconds Vbucket 117 2016-04-07T03:41:23.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37560 Milliseconds Vbucket 117 2016-04-07T03:41:23.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37590 Milliseconds Vbucket 117 2016-04-07T03:41:23.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37620 Milliseconds Vbucket 117 2016-04-07T03:41:23.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37650 Milliseconds Vbucket 117 2016-04-07T03:41:23.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37680 Milliseconds Vbucket 117 2016-04-07T03:41:23.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37710 Milliseconds Vbucket 117 2016-04-07T03:41:23.425-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3260000 QueuedCount 50000 2016-04-07T03:41:23.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37740 Milliseconds Vbucket 117 2016-04-07T03:41:23.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37770 Milliseconds Vbucket 117 2016-04-07T03:41:23.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37800 Milliseconds Vbucket 117 2016-04-07T03:41:23.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37830 Milliseconds Vbucket 117 2016-04-07T03:41:23.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37860 Milliseconds Vbucket 117 2016-04-07T03:41:23.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37890 Milliseconds Vbucket 117 2016-04-07T03:41:23.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37920 Milliseconds Vbucket 117 2016-04-07T03:41:23.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37950 Milliseconds Vbucket 117 2016-04-07T03:41:23.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37980 Milliseconds Vbucket 117 2016-04-07T03:41:23.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38010 Milliseconds Vbucket 117 2016-04-07T03:41:23.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38040 Milliseconds Vbucket 117 2016-04-07T03:41:23.746-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3270000 QueuedCount 50000 2016-04-07T03:41:23.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38070 Milliseconds Vbucket 117 2016-04-07T03:41:23.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38100 Milliseconds Vbucket 117 2016-04-07T03:41:23.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38130 Milliseconds Vbucket 117 2016-04-07T03:41:23.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38160 Milliseconds Vbucket 117 2016-04-07T03:41:23.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38190 Milliseconds Vbucket 117 2016-04-07T03:41:23.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38220 Milliseconds Vbucket 117 2016-04-07T03:41:23.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38250 Milliseconds Vbucket 117 2016-04-07T03:41:23.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38280 Milliseconds Vbucket 117 2016-04-07T03:41:24.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38310 Milliseconds Vbucket 117 2016-04-07T03:41:24.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38340 Milliseconds Vbucket 117 2016-04-07T03:41:24.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38370 Milliseconds Vbucket 117 2016-04-07T03:41:24.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38400 Milliseconds Vbucket 117 2016-04-07T03:41:24.107-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3280000 QueuedCount 50000 2016-04-07T03:41:24.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38430 Milliseconds Vbucket 117 2016-04-07T03:41:24.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38460 Milliseconds Vbucket 117 2016-04-07T03:41:24.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38490 Milliseconds Vbucket 117 2016-04-07T03:41:24.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38520 Milliseconds Vbucket 117 2016-04-07T03:41:24.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38550 Milliseconds Vbucket 117 2016-04-07T03:41:24.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38580 Milliseconds Vbucket 117 2016-04-07T03:41:24.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38610 Milliseconds Vbucket 117 2016-04-07T03:41:24.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38640 Milliseconds Vbucket 117 2016-04-07T03:41:24.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38670 Milliseconds Vbucket 117 2016-04-07T03:41:24.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38700 Milliseconds Vbucket 117 2016-04-07T03:41:24.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38730 Milliseconds Vbucket 117 2016-04-07T03:41:24.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38760 Milliseconds Vbucket 117 2016-04-07T03:41:24.469-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3290000 QueuedCount 48714 2016-04-07T03:41:24.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38790 Milliseconds Vbucket 117 2016-04-07T03:41:24.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38820 Milliseconds Vbucket 117 2016-04-07T03:41:24.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38850 Milliseconds Vbucket 117 2016-04-07T03:41:24.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38880 Milliseconds Vbucket 117 2016-04-07T03:41:24.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38910 Milliseconds Vbucket 117 2016-04-07T03:41:24.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38940 Milliseconds Vbucket 117 2016-04-07T03:41:24.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38970 Milliseconds Vbucket 117 2016-04-07T03:41:24.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39000 Milliseconds Vbucket 117 2016-04-07T03:41:24.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39030 Milliseconds Vbucket 117 2016-04-07T03:41:24.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39060 Milliseconds Vbucket 117 2016-04-07T03:41:24.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39090 Milliseconds Vbucket 117 2016-04-07T03:41:24.813-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3300000 QueuedCount 38714 2016-04-07T03:41:24.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39120 Milliseconds Vbucket 117 2016-04-07T03:41:24.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39150 Milliseconds Vbucket 117 2016-04-07T03:41:24.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39180 Milliseconds Vbucket 117 2016-04-07T03:41:24.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39210 Milliseconds Vbucket 117 2016-04-07T03:41:24.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39240 Milliseconds Vbucket 117 2016-04-07T03:41:24.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39270 Milliseconds Vbucket 117 2016-04-07T03:41:24.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39300 Milliseconds Vbucket 117 2016-04-07T03:41:25.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39330 Milliseconds Vbucket 117 2016-04-07T03:41:25.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39360 Milliseconds Vbucket 117 2016-04-07T03:41:25.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39390 Milliseconds Vbucket 117 2016-04-07T03:41:25.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39420 Milliseconds Vbucket 117 2016-04-07T03:41:25.138-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3310000 QueuedCount 28714 2016-04-07T03:41:25.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39450 Milliseconds Vbucket 117 2016-04-07T03:41:25.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39480 Milliseconds Vbucket 117 2016-04-07T03:41:25.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39510 Milliseconds Vbucket 117 2016-04-07T03:41:25.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39540 Milliseconds Vbucket 117 2016-04-07T03:41:25.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39570 Milliseconds Vbucket 117 2016-04-07T03:41:25.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39600 Milliseconds Vbucket 117 2016-04-07T03:41:25.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39630 Milliseconds Vbucket 117 2016-04-07T03:41:25.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39660 Milliseconds Vbucket 117 2016-04-07T03:41:25.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39690 Milliseconds Vbucket 117 2016-04-07T03:41:25.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39720 Milliseconds Vbucket 117 2016-04-07T03:41:25.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39750 Milliseconds Vbucket 117 2016-04-07T03:41:25.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39780 Milliseconds Vbucket 117 2016-04-07T03:41:25.479-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3320000 QueuedCount 18714 2016-04-07T03:41:25.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39810 Milliseconds Vbucket 117 2016-04-07T03:41:25.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39840 Milliseconds Vbucket 117 2016-04-07T03:41:25.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39870 Milliseconds Vbucket 117 2016-04-07T03:41:25.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39900 Milliseconds Vbucket 117 2016-04-07T03:41:25.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39930 Milliseconds Vbucket 117 2016-04-07T03:41:25.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39960 Milliseconds Vbucket 117 2016-04-07T03:41:25.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39990 Milliseconds Vbucket 117 2016-04-07T03:41:25.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40020 Milliseconds Vbucket 117 2016-04-07T03:41:25.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40050 Milliseconds Vbucket 117 2016-04-07T03:41:25.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40080 Milliseconds Vbucket 117 2016-04-07T03:41:25.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40110 Milliseconds Vbucket 117 2016-04-07T03:41:25.819-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3330000 QueuedCount 8714 2016-04-07T03:41:25.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40140 Milliseconds Vbucket 117 2016-04-07T03:41:25.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40170 Milliseconds Vbucket 117 2016-04-07T03:41:25.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40200 Milliseconds Vbucket 117 2016-04-07T03:41:25.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40230 Milliseconds Vbucket 117 2016-04-07T03:41:25.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40260 Milliseconds Vbucket 117 2016-04-07T03:41:25.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40290 Milliseconds Vbucket 117 2016-04-07T03:41:26.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40320 Milliseconds Vbucket 117 2016-04-07T03:41:26.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40350 Milliseconds Vbucket 117 2016-04-07T03:41:26.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40380 Milliseconds Vbucket 117 2016-04-07T03:41:26.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40410 Milliseconds Vbucket 117 2016-04-07T03:41:26.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40440 Milliseconds Vbucket 117 2016-04-07T03:41:26.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40470 Milliseconds Vbucket 117 2016-04-07T03:41:26.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40500 Milliseconds Vbucket 117 2016-04-07T03:41:26.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40530 Milliseconds Vbucket 117 2016-04-07T03:41:26.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40560 Milliseconds Vbucket 117 2016-04-07T03:41:26.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40590 Milliseconds Vbucket 117 2016-04-07T03:41:26.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40620 Milliseconds Vbucket 117 2016-04-07T03:41:26.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40650 Milliseconds Vbucket 117 2016-04-07T03:41:26.351-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.499µs CommitTime 266.582332ms TotalFlushTime 1m46.805013423s TotalCommitTime 3.616585822s 2016-04-07T03:41:26.351-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 3338714, 3338714, 19 committed:true 2016-04-07T03:41:26.351-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 3338714, 3338714, 19 committed:true) SnapCreateDur 276.311777ms SnapOpenDur 32.139µs 2016-04-07T03:41:26.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40680 Milliseconds Vbucket 117 2016-04-07T03:41:26.393-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3340000 QueuedCount 32096 2016-04-07T03:41:26.393-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:26.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40710 Milliseconds Vbucket 117 2016-04-07T03:41:26.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40740 Milliseconds Vbucket 117 2016-04-07T03:41:26.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40770 Milliseconds Vbucket 117 2016-04-07T03:41:26.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40800 Milliseconds Vbucket 117 2016-04-07T03:41:26.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40830 Milliseconds Vbucket 117 2016-04-07T03:41:26.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40860 Milliseconds Vbucket 117 2016-04-07T03:41:26.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40890 Milliseconds Vbucket 117 2016-04-07T03:41:26.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40920 Milliseconds Vbucket 117 2016-04-07T03:41:26.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40950 Milliseconds Vbucket 117 2016-04-07T03:41:26.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40980 Milliseconds Vbucket 117 2016-04-07T03:41:26.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41010 Milliseconds Vbucket 117 2016-04-07T03:41:26.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41040 Milliseconds Vbucket 117 2016-04-07T03:41:26.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41070 Milliseconds Vbucket 117 2016-04-07T03:41:26.774-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3350000 QueuedCount 50000 2016-04-07T03:41:26.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41100 Milliseconds Vbucket 117 2016-04-07T03:41:26.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41130 Milliseconds Vbucket 117 2016-04-07T03:41:26.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41160 Milliseconds Vbucket 117 2016-04-07T03:41:26.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41190 Milliseconds Vbucket 117 2016-04-07T03:41:26.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41220 Milliseconds Vbucket 117 2016-04-07T03:41:26.93-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:26.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41250 Milliseconds Vbucket 117 2016-04-07T03:41:26.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41280 Milliseconds Vbucket 117 2016-04-07T03:41:27.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41310 Milliseconds Vbucket 117 2016-04-07T03:41:27.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41340 Milliseconds Vbucket 117 2016-04-07T03:41:27.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41370 Milliseconds Vbucket 117 2016-04-07T03:41:27.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41400 Milliseconds Vbucket 117 2016-04-07T03:41:27.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41430 Milliseconds Vbucket 117 2016-04-07T03:41:27.146-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3360000 QueuedCount 50000 2016-04-07T03:41:27.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41460 Milliseconds Vbucket 117 2016-04-07T03:41:27.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41490 Milliseconds Vbucket 117 2016-04-07T03:41:27.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41520 Milliseconds Vbucket 117 2016-04-07T03:41:27.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41550 Milliseconds Vbucket 117 2016-04-07T03:41:27.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41580 Milliseconds Vbucket 117 2016-04-07T03:41:27.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41610 Milliseconds Vbucket 117 2016-04-07T03:41:27.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41640 Milliseconds Vbucket 117 2016-04-07T03:41:27.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41670 Milliseconds Vbucket 117 2016-04-07T03:41:27.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41700 Milliseconds Vbucket 117 2016-04-07T03:41:27.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41730 Milliseconds Vbucket 117 2016-04-07T03:41:27.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41760 Milliseconds Vbucket 117 2016-04-07T03:41:27.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41790 Milliseconds Vbucket 117 2016-04-07T03:41:27.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41820 Milliseconds Vbucket 117 2016-04-07T03:41:27.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41850 Milliseconds Vbucket 117 2016-04-07T03:41:27.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41880 Milliseconds Vbucket 117 2016-04-07T03:41:27.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41910 Milliseconds Vbucket 117 2016-04-07T03:41:27.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41940 Milliseconds Vbucket 117 2016-04-07T03:41:27.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41970 Milliseconds Vbucket 117 2016-04-07T03:41:27.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42000 Milliseconds Vbucket 117 2016-04-07T03:41:27.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42030 Milliseconds Vbucket 117 2016-04-07T03:41:27.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42060 Milliseconds Vbucket 117 2016-04-07T03:41:27.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42090 Milliseconds Vbucket 117 2016-04-07T03:41:27.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42120 Milliseconds Vbucket 117 2016-04-07T03:41:27.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42150 Milliseconds Vbucket 117 2016-04-07T03:41:27.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42180 Milliseconds Vbucket 117 2016-04-07T03:41:27.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42210 Milliseconds Vbucket 117 2016-04-07T03:41:27.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42240 Milliseconds Vbucket 117 2016-04-07T03:41:27.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42270 Milliseconds Vbucket 117 2016-04-07T03:41:27.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42300 Milliseconds Vbucket 117 2016-04-07T03:41:28.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42330 Milliseconds Vbucket 117 2016-04-07T03:41:28.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42360 Milliseconds Vbucket 117 2016-04-07T03:41:28.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42390 Milliseconds Vbucket 117 2016-04-07T03:41:28.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42420 Milliseconds Vbucket 117 2016-04-07T03:41:28.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42450 Milliseconds Vbucket 117 2016-04-07T03:41:28.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42480 Milliseconds Vbucket 117 2016-04-07T03:41:28.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42510 Milliseconds Vbucket 117 2016-04-07T03:41:28.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42540 Milliseconds Vbucket 117 2016-04-07T03:41:28.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42570 Milliseconds Vbucket 117 2016-04-07T03:41:28.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42600 Milliseconds Vbucket 117 2016-04-07T03:41:28.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42630 Milliseconds Vbucket 117 2016-04-07T03:41:28.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42660 Milliseconds Vbucket 117 2016-04-07T03:41:28.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42690 Milliseconds Vbucket 117 2016-04-07T03:41:28.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42720 Milliseconds Vbucket 117 2016-04-07T03:41:28.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42750 Milliseconds Vbucket 117 2016-04-07T03:41:28.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42780 Milliseconds Vbucket 117 2016-04-07T03:41:28.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42810 Milliseconds Vbucket 117 2016-04-07T03:41:28.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42840 Milliseconds Vbucket 117 2016-04-07T03:41:28.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42870 Milliseconds Vbucket 117 2016-04-07T03:41:28.59-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3370000 QueuedCount 50000 2016-04-07T03:41:28.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42900 Milliseconds Vbucket 117 2016-04-07T03:41:28.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42930 Milliseconds Vbucket 117 2016-04-07T03:41:28.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42960 Milliseconds Vbucket 117 2016-04-07T03:41:28.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42990 Milliseconds Vbucket 117 2016-04-07T03:41:28.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43020 Milliseconds Vbucket 117 2016-04-07T03:41:28.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43050 Milliseconds Vbucket 117 2016-04-07T03:41:28.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43080 Milliseconds Vbucket 117 2016-04-07T03:41:28.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43110 Milliseconds Vbucket 117 2016-04-07T03:41:28.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43140 Milliseconds Vbucket 117 2016-04-07T03:41:28.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43170 Milliseconds Vbucket 117 2016-04-07T03:41:28.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43200 Milliseconds Vbucket 117 2016-04-07T03:41:28.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43230 Milliseconds Vbucket 117 2016-04-07T03:41:28.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43260 Milliseconds Vbucket 117 2016-04-07T03:41:28.975-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3380000 QueuedCount 50000 2016-04-07T03:41:28.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43290 Milliseconds Vbucket 117 2016-04-07T03:41:29.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43320 Milliseconds Vbucket 117 2016-04-07T03:41:29.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43350 Milliseconds Vbucket 117 2016-04-07T03:41:29.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43380 Milliseconds Vbucket 117 2016-04-07T03:41:29.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43410 Milliseconds Vbucket 117 2016-04-07T03:41:29.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43440 Milliseconds Vbucket 117 2016-04-07T03:41:29.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43470 Milliseconds Vbucket 117 2016-04-07T03:41:29.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43500 Milliseconds Vbucket 117 2016-04-07T03:41:29.208-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:29.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43530 Milliseconds Vbucket 117 2016-04-07T03:41:29.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43560 Milliseconds Vbucket 117 2016-04-07T03:41:29.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43590 Milliseconds Vbucket 117 2016-04-07T03:41:29.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43620 Milliseconds Vbucket 117 2016-04-07T03:41:29.323-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3390000 QueuedCount 50000 2016-04-07T03:41:29.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43650 Milliseconds Vbucket 117 2016-04-07T03:41:29.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43680 Milliseconds Vbucket 117 2016-04-07T03:41:29.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43710 Milliseconds Vbucket 117 2016-04-07T03:41:29.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43740 Milliseconds Vbucket 117 2016-04-07T03:41:29.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43770 Milliseconds Vbucket 117 2016-04-07T03:41:29.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43800 Milliseconds Vbucket 117 2016-04-07T03:41:29.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43830 Milliseconds Vbucket 117 2016-04-07T03:41:29.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43860 Milliseconds Vbucket 117 2016-04-07T03:41:29.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43890 Milliseconds Vbucket 117 2016-04-07T03:41:29.601-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3400000 QueuedCount 50000 2016-04-07T03:41:29.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43920 Milliseconds Vbucket 117 2016-04-07T03:41:29.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43950 Milliseconds Vbucket 117 2016-04-07T03:41:29.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43980 Milliseconds Vbucket 117 2016-04-07T03:41:29.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44010 Milliseconds Vbucket 117 2016-04-07T03:41:29.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44040 Milliseconds Vbucket 117 2016-04-07T03:41:29.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44070 Milliseconds Vbucket 117 2016-04-07T03:41:29.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44100 Milliseconds Vbucket 117 2016-04-07T03:41:29.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44130 Milliseconds Vbucket 117 2016-04-07T03:41:29.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44160 Milliseconds Vbucket 117 2016-04-07T03:41:29.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44190 Milliseconds Vbucket 117 2016-04-07T03:41:29.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44220 Milliseconds Vbucket 117 2016-04-07T03:41:29.921-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3410000 QueuedCount 50000 2016-04-07T03:41:29.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44250 Milliseconds Vbucket 117 2016-04-07T03:41:29.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44280 Milliseconds Vbucket 117 2016-04-07T03:41:30.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44310 Milliseconds Vbucket 117 2016-04-07T03:41:30.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44340 Milliseconds Vbucket 117 2016-04-07T03:41:30.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44370 Milliseconds Vbucket 117 2016-04-07T03:41:30.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44400 Milliseconds Vbucket 117 2016-04-07T03:41:30.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44430 Milliseconds Vbucket 117 2016-04-07T03:41:30.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44460 Milliseconds Vbucket 117 2016-04-07T03:41:30.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44490 Milliseconds Vbucket 117 2016-04-07T03:41:30.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44520 Milliseconds Vbucket 117 2016-04-07T03:41:30.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44550 Milliseconds Vbucket 117 2016-04-07T03:41:30.244-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3420000 QueuedCount 50000 2016-04-07T03:41:30.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44580 Milliseconds Vbucket 117 2016-04-07T03:41:30.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44610 Milliseconds Vbucket 117 2016-04-07T03:41:30.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44640 Milliseconds Vbucket 117 2016-04-07T03:41:30.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44670 Milliseconds Vbucket 117 2016-04-07T03:41:30.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44700 Milliseconds Vbucket 117 2016-04-07T03:41:30.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44730 Milliseconds Vbucket 117 2016-04-07T03:41:30.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44760 Milliseconds Vbucket 117 2016-04-07T03:41:30.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44790 Milliseconds Vbucket 117 2016-04-07T03:41:30.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44820 Milliseconds Vbucket 117 2016-04-07T03:41:30.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44850 Milliseconds Vbucket 117 2016-04-07T03:41:30.561-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3430000 QueuedCount 40159 2016-04-07T03:41:30.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44880 Milliseconds Vbucket 117 2016-04-07T03:41:30.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44910 Milliseconds Vbucket 117 2016-04-07T03:41:30.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44940 Milliseconds Vbucket 117 2016-04-07T03:41:30.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44970 Milliseconds Vbucket 117 2016-04-07T03:41:30.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45000 Milliseconds Vbucket 117 2016-04-07T03:41:30.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45030 Milliseconds Vbucket 117 2016-04-07T03:41:30.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45060 Milliseconds Vbucket 117 2016-04-07T03:41:30.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45090 Milliseconds Vbucket 117 2016-04-07T03:41:30.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45120 Milliseconds Vbucket 117 2016-04-07T03:41:30.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45150 Milliseconds Vbucket 117 2016-04-07T03:41:30.87-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3440000 QueuedCount 30159 2016-04-07T03:41:30.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45180 Milliseconds Vbucket 117 2016-04-07T03:41:30.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45210 Milliseconds Vbucket 117 2016-04-07T03:41:30.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45240 Milliseconds Vbucket 117 2016-04-07T03:41:30.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45270 Milliseconds Vbucket 117 2016-04-07T03:41:30.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45300 Milliseconds Vbucket 117 2016-04-07T03:41:31.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45330 Milliseconds Vbucket 117 2016-04-07T03:41:31.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45360 Milliseconds Vbucket 117 2016-04-07T03:41:31.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45390 Milliseconds Vbucket 117 2016-04-07T03:41:31.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45420 Milliseconds Vbucket 117 2016-04-07T03:41:31.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45450 Milliseconds Vbucket 117 2016-04-07T03:41:31.15-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3450000 QueuedCount 20159 2016-04-07T03:41:31.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45480 Milliseconds Vbucket 117 2016-04-07T03:41:31.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45510 Milliseconds Vbucket 117 2016-04-07T03:41:31.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45540 Milliseconds Vbucket 117 2016-04-07T03:41:31.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45570 Milliseconds Vbucket 117 2016-04-07T03:41:31.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45600 Milliseconds Vbucket 117 2016-04-07T03:41:31.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45630 Milliseconds Vbucket 117 2016-04-07T03:41:31.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45660 Milliseconds Vbucket 117 2016-04-07T03:41:31.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45690 Milliseconds Vbucket 117 2016-04-07T03:41:31.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45720 Milliseconds Vbucket 117 2016-04-07T03:41:31.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45750 Milliseconds Vbucket 117 2016-04-07T03:41:31.446-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3460000 QueuedCount 10159 2016-04-07T03:41:31.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45780 Milliseconds Vbucket 117 2016-04-07T03:41:31.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45810 Milliseconds Vbucket 117 2016-04-07T03:41:31.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45840 Milliseconds Vbucket 117 2016-04-07T03:41:31.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45870 Milliseconds Vbucket 117 2016-04-07T03:41:31.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45900 Milliseconds Vbucket 117 2016-04-07T03:41:31.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45930 Milliseconds Vbucket 117 2016-04-07T03:41:31.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45960 Milliseconds Vbucket 117 2016-04-07T03:41:31.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45990 Milliseconds Vbucket 117 2016-04-07T03:41:31.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46020 Milliseconds Vbucket 117 2016-04-07T03:41:31.738-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3470000 QueuedCount 159 2016-04-07T03:41:31.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46050 Milliseconds Vbucket 117 2016-04-07T03:41:31.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46080 Milliseconds Vbucket 117 2016-04-07T03:41:31.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46110 Milliseconds Vbucket 117 2016-04-07T03:41:31.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46140 Milliseconds Vbucket 117 2016-04-07T03:41:31.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46170 Milliseconds Vbucket 117 2016-04-07T03:41:31.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46200 Milliseconds Vbucket 117 2016-04-07T03:41:31.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46230 Milliseconds Vbucket 117 2016-04-07T03:41:31.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46260 Milliseconds Vbucket 117 2016-04-07T03:41:31.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46290 Milliseconds Vbucket 117 2016-04-07T03:41:32.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46320 Milliseconds Vbucket 117 2016-04-07T03:41:32.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46350 Milliseconds Vbucket 117 2016-04-07T03:41:32.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46380 Milliseconds Vbucket 117 2016-04-07T03:41:32.087-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.546µs CommitTime 330.171075ms TotalFlushTime 1m51.977851746s TotalCommitTime 3.946756897s 2016-04-07T03:41:32.087-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 3470159, 3470159, 20 committed:true 2016-04-07T03:41:32.087-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 3470159, 3470159, 20 committed:true) SnapCreateDur 340.020667ms SnapOpenDur 48.468µs 2016-04-07T03:41:32.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46410 Milliseconds Vbucket 117 2016-04-07T03:41:32.127-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:32.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46440 Milliseconds Vbucket 117 2016-04-07T03:41:32.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46470 Milliseconds Vbucket 117 2016-04-07T03:41:32.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46500 Milliseconds Vbucket 117 2016-04-07T03:41:32.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46530 Milliseconds Vbucket 117 2016-04-07T03:41:32.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46560 Milliseconds Vbucket 117 2016-04-07T03:41:32.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46590 Milliseconds Vbucket 117 2016-04-07T03:41:32.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46620 Milliseconds Vbucket 117 2016-04-07T03:41:32.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46650 Milliseconds Vbucket 117 2016-04-07T03:41:32.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46680 Milliseconds Vbucket 117 2016-04-07T03:41:32.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46710 Milliseconds Vbucket 117 2016-04-07T03:41:32.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46740 Milliseconds Vbucket 117 2016-04-07T03:41:32.459-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3480000 QueuedCount 50000 2016-04-07T03:41:32.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46770 Milliseconds Vbucket 117 2016-04-07T03:41:32.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46800 Milliseconds Vbucket 117 2016-04-07T03:41:32.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46830 Milliseconds Vbucket 117 2016-04-07T03:41:32.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46860 Milliseconds Vbucket 117 2016-04-07T03:41:32.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46890 Milliseconds Vbucket 117 2016-04-07T03:41:32.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46920 Milliseconds Vbucket 117 2016-04-07T03:41:32.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46950 Milliseconds Vbucket 117 2016-04-07T03:41:32.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46980 Milliseconds Vbucket 117 2016-04-07T03:41:32.679-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:32.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47010 Milliseconds Vbucket 117 2016-04-07T03:41:32.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47040 Milliseconds Vbucket 117 2016-04-07T03:41:32.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47070 Milliseconds Vbucket 117 2016-04-07T03:41:32.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47100 Milliseconds Vbucket 117 2016-04-07T03:41:32.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47130 Milliseconds Vbucket 117 2016-04-07T03:41:32.841-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3490000 QueuedCount 50000 2016-04-07T03:41:32.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47160 Milliseconds Vbucket 117 2016-04-07T03:41:32.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47190 Milliseconds Vbucket 117 2016-04-07T03:41:32.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47220 Milliseconds Vbucket 117 2016-04-07T03:41:32.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47250 Milliseconds Vbucket 117 2016-04-07T03:41:32.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47280 Milliseconds Vbucket 117 2016-04-07T03:41:33.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47310 Milliseconds Vbucket 117 2016-04-07T03:41:33.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47340 Milliseconds Vbucket 117 2016-04-07T03:41:33.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47370 Milliseconds Vbucket 117 2016-04-07T03:41:33.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47400 Milliseconds Vbucket 117 2016-04-07T03:41:33.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47430 Milliseconds Vbucket 117 2016-04-07T03:41:33.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47460 Milliseconds Vbucket 117 2016-04-07T03:41:33.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47490 Milliseconds Vbucket 117 2016-04-07T03:41:33.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47520 Milliseconds Vbucket 117 2016-04-07T03:41:33.228-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3500000 QueuedCount 50000 2016-04-07T03:41:33.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47550 Milliseconds Vbucket 117 2016-04-07T03:41:33.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47580 Milliseconds Vbucket 117 2016-04-07T03:41:33.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47610 Milliseconds Vbucket 117 2016-04-07T03:41:33.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47640 Milliseconds Vbucket 117 2016-04-07T03:41:33.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47670 Milliseconds Vbucket 117 2016-04-07T03:41:33.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47700 Milliseconds Vbucket 117 2016-04-07T03:41:33.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47730 Milliseconds Vbucket 117 2016-04-07T03:41:33.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47760 Milliseconds Vbucket 117 2016-04-07T03:41:33.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47790 Milliseconds Vbucket 117 2016-04-07T03:41:33.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47820 Milliseconds Vbucket 117 2016-04-07T03:41:33.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47850 Milliseconds Vbucket 117 2016-04-07T03:41:33.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47880 Milliseconds Vbucket 117 2016-04-07T03:41:33.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47910 Milliseconds Vbucket 117 2016-04-07T03:41:33.623-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3510000 QueuedCount 50000 2016-04-07T03:41:33.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47940 Milliseconds Vbucket 117 2016-04-07T03:41:33.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47970 Milliseconds Vbucket 117 2016-04-07T03:41:33.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48000 Milliseconds Vbucket 117 2016-04-07T03:41:33.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48030 Milliseconds Vbucket 117 2016-04-07T03:41:33.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48060 Milliseconds Vbucket 117 2016-04-07T03:41:33.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48090 Milliseconds Vbucket 117 2016-04-07T03:41:33.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48120 Milliseconds Vbucket 117 2016-04-07T03:41:33.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48150 Milliseconds Vbucket 117 2016-04-07T03:41:33.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48180 Milliseconds Vbucket 117 2016-04-07T03:41:33.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48210 Milliseconds Vbucket 117 2016-04-07T03:41:33.931-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:33.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48240 Milliseconds Vbucket 117 2016-04-07T03:41:33.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48270 Milliseconds Vbucket 117 2016-04-07T03:41:33.988-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3520000 QueuedCount 50000 2016-04-07T03:41:33.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48300 Milliseconds Vbucket 117 2016-04-07T03:41:34.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48330 Milliseconds Vbucket 117 2016-04-07T03:41:34.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48360 Milliseconds Vbucket 117 2016-04-07T03:41:34.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48390 Milliseconds Vbucket 117 2016-04-07T03:41:34.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48420 Milliseconds Vbucket 117 2016-04-07T03:41:34.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48450 Milliseconds Vbucket 117 2016-04-07T03:41:34.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48480 Milliseconds Vbucket 117 2016-04-07T03:41:34.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48510 Milliseconds Vbucket 117 2016-04-07T03:41:34.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48540 Milliseconds Vbucket 117 2016-04-07T03:41:34.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48570 Milliseconds Vbucket 117 2016-04-07T03:41:34.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48600 Milliseconds Vbucket 117 2016-04-07T03:41:34.311-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3530000 QueuedCount 50000 2016-04-07T03:41:34.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48630 Milliseconds Vbucket 117 2016-04-07T03:41:34.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48660 Milliseconds Vbucket 117 2016-04-07T03:41:34.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48690 Milliseconds Vbucket 117 2016-04-07T03:41:34.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48720 Milliseconds Vbucket 117 2016-04-07T03:41:34.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48750 Milliseconds Vbucket 117 2016-04-07T03:41:34.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48780 Milliseconds Vbucket 117 2016-04-07T03:41:34.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48810 Milliseconds Vbucket 117 2016-04-07T03:41:34.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48840 Milliseconds Vbucket 117 2016-04-07T03:41:34.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48870 Milliseconds Vbucket 117 2016-04-07T03:41:34.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48900 Milliseconds Vbucket 117 2016-04-07T03:41:34.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48930 Milliseconds Vbucket 117 2016-04-07T03:41:34.644-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3540000 QueuedCount 50000 2016-04-07T03:41:34.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48960 Milliseconds Vbucket 117 2016-04-07T03:41:34.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48990 Milliseconds Vbucket 117 2016-04-07T03:41:34.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49020 Milliseconds Vbucket 117 2016-04-07T03:41:34.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49050 Milliseconds Vbucket 117 2016-04-07T03:41:34.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49080 Milliseconds Vbucket 117 2016-04-07T03:41:34.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49110 Milliseconds Vbucket 117 2016-04-07T03:41:34.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49140 Milliseconds Vbucket 117 2016-04-07T03:41:34.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49170 Milliseconds Vbucket 117 2016-04-07T03:41:34.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49200 Milliseconds Vbucket 117 2016-04-07T03:41:34.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49230 Milliseconds Vbucket 117 2016-04-07T03:41:34.929-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3550000 QueuedCount 50000 2016-04-07T03:41:34.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49260 Milliseconds Vbucket 117 2016-04-07T03:41:34.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49290 Milliseconds Vbucket 117 2016-04-07T03:41:35.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49320 Milliseconds Vbucket 117 2016-04-07T03:41:35.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49350 Milliseconds Vbucket 117 2016-04-07T03:41:35.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49380 Milliseconds Vbucket 117 2016-04-07T03:41:35.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49410 Milliseconds Vbucket 117 2016-04-07T03:41:35.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49440 Milliseconds Vbucket 117 2016-04-07T03:41:35.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49470 Milliseconds Vbucket 117 2016-04-07T03:41:35.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49500 Milliseconds Vbucket 117 2016-04-07T03:41:35.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49530 Milliseconds Vbucket 117 2016-04-07T03:41:35.246-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3560000 QueuedCount 50000 2016-04-07T03:41:35.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49560 Milliseconds Vbucket 117 2016-04-07T03:41:35.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49590 Milliseconds Vbucket 117 2016-04-07T03:41:35.307-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:35.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49620 Milliseconds Vbucket 117 2016-04-07T03:41:35.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49650 Milliseconds Vbucket 117 2016-04-07T03:41:35.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49680 Milliseconds Vbucket 117 2016-04-07T03:41:35.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49710 Milliseconds Vbucket 117 2016-04-07T03:41:35.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49740 Milliseconds Vbucket 117 2016-04-07T03:41:35.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49770 Milliseconds Vbucket 117 2016-04-07T03:41:35.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49800 Milliseconds Vbucket 117 2016-04-07T03:41:35.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49830 Milliseconds Vbucket 117 2016-04-07T03:41:35.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49860 Milliseconds Vbucket 117 2016-04-07T03:41:35.568-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3570000 QueuedCount 50000 2016-04-07T03:41:35.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49890 Milliseconds Vbucket 117 2016-04-07T03:41:35.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49920 Milliseconds Vbucket 117 2016-04-07T03:41:35.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49950 Milliseconds Vbucket 117 2016-04-07T03:41:35.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49980 Milliseconds Vbucket 117 2016-04-07T03:41:35.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50010 Milliseconds Vbucket 117 2016-04-07T03:41:35.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50040 Milliseconds Vbucket 117 2016-04-07T03:41:35.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50070 Milliseconds Vbucket 117 2016-04-07T03:41:35.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50100 Milliseconds Vbucket 117 2016-04-07T03:41:35.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50130 Milliseconds Vbucket 117 2016-04-07T03:41:35.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50160 Milliseconds Vbucket 117 2016-04-07T03:41:35.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50190 Milliseconds Vbucket 117 2016-04-07T03:41:35.887-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3580000 QueuedCount 50000 2016-04-07T03:41:35.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50220 Milliseconds Vbucket 117 2016-04-07T03:41:35.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50250 Milliseconds Vbucket 117 2016-04-07T03:41:35.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50280 Milliseconds Vbucket 117 2016-04-07T03:41:36.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50310 Milliseconds Vbucket 117 2016-04-07T03:41:36.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50340 Milliseconds Vbucket 117 2016-04-07T03:41:36.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50370 Milliseconds Vbucket 117 2016-04-07T03:41:36.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50400 Milliseconds Vbucket 117 2016-04-07T03:41:36.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50430 Milliseconds Vbucket 117 2016-04-07T03:41:36.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50460 Milliseconds Vbucket 117 2016-04-07T03:41:36.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50490 Milliseconds Vbucket 117 2016-04-07T03:41:36.192-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3590000 QueuedCount 50000 2016-04-07T03:41:36.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50520 Milliseconds Vbucket 117 2016-04-07T03:41:36.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50550 Milliseconds Vbucket 117 2016-04-07T03:41:36.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50580 Milliseconds Vbucket 117 2016-04-07T03:41:36.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50610 Milliseconds Vbucket 117 2016-04-07T03:41:36.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50640 Milliseconds Vbucket 117 2016-04-07T03:41:36.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50670 Milliseconds Vbucket 117 2016-04-07T03:41:36.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50700 Milliseconds Vbucket 117 2016-04-07T03:41:36.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50730 Milliseconds Vbucket 117 2016-04-07T03:41:36.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50760 Milliseconds Vbucket 117 2016-04-07T03:41:36.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50790 Milliseconds Vbucket 117 2016-04-07T03:41:36.495-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3600000 QueuedCount 50000 2016-04-07T03:41:36.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50820 Milliseconds Vbucket 117 2016-04-07T03:41:36.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50850 Milliseconds Vbucket 117 2016-04-07T03:41:36.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50880 Milliseconds Vbucket 117 2016-04-07T03:41:36.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50910 Milliseconds Vbucket 117 2016-04-07T03:41:36.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50940 Milliseconds Vbucket 117 2016-04-07T03:41:36.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50970 Milliseconds Vbucket 117 2016-04-07T03:41:36.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51000 Milliseconds Vbucket 117 2016-04-07T03:41:36.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51030 Milliseconds Vbucket 117 2016-04-07T03:41:36.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51060 Milliseconds Vbucket 117 2016-04-07T03:41:36.782-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3610000 QueuedCount 46940 2016-04-07T03:41:36.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51090 Milliseconds Vbucket 117 2016-04-07T03:41:36.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51120 Milliseconds Vbucket 117 2016-04-07T03:41:36.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51150 Milliseconds Vbucket 117 2016-04-07T03:41:36.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51180 Milliseconds Vbucket 117 2016-04-07T03:41:36.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51210 Milliseconds Vbucket 117 2016-04-07T03:41:36.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51240 Milliseconds Vbucket 117 2016-04-07T03:41:36.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51270 Milliseconds Vbucket 117 2016-04-07T03:41:36.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51300 Milliseconds Vbucket 117 2016-04-07T03:41:37.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51330 Milliseconds Vbucket 117 2016-04-07T03:41:37.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51360 Milliseconds Vbucket 117 2016-04-07T03:41:37.068-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3620000 QueuedCount 36940 2016-04-07T03:41:37.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51390 Milliseconds Vbucket 117 2016-04-07T03:41:37.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51420 Milliseconds Vbucket 117 2016-04-07T03:41:37.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51450 Milliseconds Vbucket 117 2016-04-07T03:41:37.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51480 Milliseconds Vbucket 117 2016-04-07T03:41:37.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51510 Milliseconds Vbucket 117 2016-04-07T03:41:37.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51540 Milliseconds Vbucket 117 2016-04-07T03:41:37.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51570 Milliseconds Vbucket 117 2016-04-07T03:41:37.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51600 Milliseconds Vbucket 117 2016-04-07T03:41:37.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51630 Milliseconds Vbucket 117 2016-04-07T03:41:37.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51660 Milliseconds Vbucket 117 2016-04-07T03:41:37.363-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3630000 QueuedCount 26940 2016-04-07T03:41:37.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51690 Milliseconds Vbucket 117 2016-04-07T03:41:37.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51720 Milliseconds Vbucket 117 2016-04-07T03:41:37.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51750 Milliseconds Vbucket 117 2016-04-07T03:41:37.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51780 Milliseconds Vbucket 117 2016-04-07T03:41:37.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51810 Milliseconds Vbucket 117 2016-04-07T03:41:37.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51840 Milliseconds Vbucket 117 2016-04-07T03:41:37.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51870 Milliseconds Vbucket 117 2016-04-07T03:41:37.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51900 Milliseconds Vbucket 117 2016-04-07T03:41:37.608-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3640000 QueuedCount 16940 2016-04-07T03:41:37.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51930 Milliseconds Vbucket 117 2016-04-07T03:41:37.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51960 Milliseconds Vbucket 117 2016-04-07T03:41:37.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51990 Milliseconds Vbucket 117 2016-04-07T03:41:37.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52020 Milliseconds Vbucket 117 2016-04-07T03:41:37.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52050 Milliseconds Vbucket 117 2016-04-07T03:41:37.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52080 Milliseconds Vbucket 117 2016-04-07T03:41:37.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52110 Milliseconds Vbucket 117 2016-04-07T03:41:37.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52140 Milliseconds Vbucket 117 2016-04-07T03:41:37.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52170 Milliseconds Vbucket 117 2016-04-07T03:41:37.873-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3650000 QueuedCount 6940 2016-04-07T03:41:37.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52200 Milliseconds Vbucket 117 2016-04-07T03:41:37.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52230 Milliseconds Vbucket 117 2016-04-07T03:41:37.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52260 Milliseconds Vbucket 117 2016-04-07T03:41:37.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52290 Milliseconds Vbucket 117 2016-04-07T03:41:38.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52320 Milliseconds Vbucket 117 2016-04-07T03:41:38.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52350 Milliseconds Vbucket 117 2016-04-07T03:41:38.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52380 Milliseconds Vbucket 117 2016-04-07T03:41:38.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52410 Milliseconds Vbucket 117 2016-04-07T03:41:38.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52440 Milliseconds Vbucket 117 2016-04-07T03:41:38.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52470 Milliseconds Vbucket 117 2016-04-07T03:41:38.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52500 Milliseconds Vbucket 117 2016-04-07T03:41:38.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52530 Milliseconds Vbucket 117 2016-04-07T03:41:38.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52560 Milliseconds Vbucket 117 2016-04-07T03:41:38.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52590 Milliseconds Vbucket 117 2016-04-07T03:41:38.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52620 Milliseconds Vbucket 117 2016-04-07T03:41:38.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52650 Milliseconds Vbucket 117 2016-04-07T03:41:38.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52680 Milliseconds Vbucket 117 2016-04-07T03:41:38.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52710 Milliseconds Vbucket 117 2016-04-07T03:41:38.425-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.21µs CommitTime 324.013869ms TotalFlushTime 1m57.657121979s TotalCommitTime 4.270770766s 2016-04-07T03:41:38.425-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 3656940, 3656940, 21 committed:true 2016-04-07T03:41:38.425-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 3656940, 3656940, 21 committed:true) SnapCreateDur 334.569366ms SnapOpenDur 40.647µs 2016-04-07T03:41:38.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52740 Milliseconds Vbucket 117 2016-04-07T03:41:38.46-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:38.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52770 Milliseconds Vbucket 117 2016-04-07T03:41:38.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52800 Milliseconds Vbucket 117 2016-04-07T03:41:38.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52830 Milliseconds Vbucket 117 2016-04-07T03:41:38.543-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3660000 QueuedCount 50000 2016-04-07T03:41:38.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52860 Milliseconds Vbucket 117 2016-04-07T03:41:38.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52890 Milliseconds Vbucket 117 2016-04-07T03:41:38.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52920 Milliseconds Vbucket 117 2016-04-07T03:41:38.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52950 Milliseconds Vbucket 117 2016-04-07T03:41:38.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52980 Milliseconds Vbucket 117 2016-04-07T03:41:38.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53010 Milliseconds Vbucket 117 2016-04-07T03:41:38.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53040 Milliseconds Vbucket 117 2016-04-07T03:41:38.759-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:38.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53070 Milliseconds Vbucket 117 2016-04-07T03:41:38.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53100 Milliseconds Vbucket 117 2016-04-07T03:41:38.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53130 Milliseconds Vbucket 117 2016-04-07T03:41:38.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53160 Milliseconds Vbucket 117 2016-04-07T03:41:38.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53190 Milliseconds Vbucket 117 2016-04-07T03:41:38.889-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3670000 QueuedCount 50000 2016-04-07T03:41:38.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53220 Milliseconds Vbucket 117 2016-04-07T03:41:38.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53250 Milliseconds Vbucket 117 2016-04-07T03:41:38.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53280 Milliseconds Vbucket 117 2016-04-07T03:41:39.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53310 Milliseconds Vbucket 117 2016-04-07T03:41:39.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53340 Milliseconds Vbucket 117 2016-04-07T03:41:39.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53370 Milliseconds Vbucket 117 2016-04-07T03:41:39.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53400 Milliseconds Vbucket 117 2016-04-07T03:41:39.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53430 Milliseconds Vbucket 117 2016-04-07T03:41:39.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53460 Milliseconds Vbucket 117 2016-04-07T03:41:39.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53490 Milliseconds Vbucket 117 2016-04-07T03:41:39.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53520 Milliseconds Vbucket 117 2016-04-07T03:41:39.24-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3680000 QueuedCount 50000 2016-04-07T03:41:39.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53550 Milliseconds Vbucket 117 2016-04-07T03:41:39.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53580 Milliseconds Vbucket 117 2016-04-07T03:41:39.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53610 Milliseconds Vbucket 117 2016-04-07T03:41:39.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53640 Milliseconds Vbucket 117 2016-04-07T03:41:39.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53670 Milliseconds Vbucket 117 2016-04-07T03:41:39.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53700 Milliseconds Vbucket 117 2016-04-07T03:41:39.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53730 Milliseconds Vbucket 117 2016-04-07T03:41:39.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53760 Milliseconds Vbucket 117 2016-04-07T03:41:39.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53790 Milliseconds Vbucket 117 2016-04-07T03:41:39.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53820 Milliseconds Vbucket 117 2016-04-07T03:41:39.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53850 Milliseconds Vbucket 117 2016-04-07T03:41:39.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53880 Milliseconds Vbucket 117 2016-04-07T03:41:39.575-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3690000 QueuedCount 50000 2016-04-07T03:41:39.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53910 Milliseconds Vbucket 117 2016-04-07T03:41:39.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53940 Milliseconds Vbucket 117 2016-04-07T03:41:39.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53970 Milliseconds Vbucket 117 2016-04-07T03:41:39.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54000 Milliseconds Vbucket 117 2016-04-07T03:41:39.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54030 Milliseconds Vbucket 117 2016-04-07T03:41:39.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54060 Milliseconds Vbucket 117 2016-04-07T03:41:39.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54090 Milliseconds Vbucket 117 2016-04-07T03:41:39.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54120 Milliseconds Vbucket 117 2016-04-07T03:41:39.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54150 Milliseconds Vbucket 117 2016-04-07T03:41:39.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54180 Milliseconds Vbucket 117 2016-04-07T03:41:39.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54210 Milliseconds Vbucket 117 2016-04-07T03:41:39.931-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3700000 QueuedCount 50000 2016-04-07T03:41:39.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54240 Milliseconds Vbucket 117 2016-04-07T03:41:39.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54270 Milliseconds Vbucket 117 2016-04-07T03:41:39.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54300 Milliseconds Vbucket 117 2016-04-07T03:41:39.997-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:40.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54330 Milliseconds Vbucket 117 2016-04-07T03:41:40.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54360 Milliseconds Vbucket 117 2016-04-07T03:41:40.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54390 Milliseconds Vbucket 117 2016-04-07T03:41:40.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54420 Milliseconds Vbucket 117 2016-04-07T03:41:40.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54450 Milliseconds Vbucket 117 2016-04-07T03:41:40.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54480 Milliseconds Vbucket 117 2016-04-07T03:41:40.193-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3710000 QueuedCount 50000 2016-04-07T03:41:40.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54510 Milliseconds Vbucket 117 2016-04-07T03:41:40.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54540 Milliseconds Vbucket 117 2016-04-07T03:41:40.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54570 Milliseconds Vbucket 117 2016-04-07T03:41:40.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54600 Milliseconds Vbucket 117 2016-04-07T03:41:40.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54630 Milliseconds Vbucket 117 2016-04-07T03:41:40.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54660 Milliseconds Vbucket 117 2016-04-07T03:41:40.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54690 Milliseconds Vbucket 117 2016-04-07T03:41:40.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54720 Milliseconds Vbucket 117 2016-04-07T03:41:40.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54750 Milliseconds Vbucket 117 2016-04-07T03:41:40.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54780 Milliseconds Vbucket 117 2016-04-07T03:41:40.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54810 Milliseconds Vbucket 117 2016-04-07T03:41:40.511-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3720000 QueuedCount 50000 2016-04-07T03:41:40.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54840 Milliseconds Vbucket 117 2016-04-07T03:41:40.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54870 Milliseconds Vbucket 117 2016-04-07T03:41:40.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54900 Milliseconds Vbucket 117 2016-04-07T03:41:40.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54930 Milliseconds Vbucket 117 2016-04-07T03:41:40.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54960 Milliseconds Vbucket 117 2016-04-07T03:41:40.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54990 Milliseconds Vbucket 117 2016-04-07T03:41:40.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55020 Milliseconds Vbucket 117 2016-04-07T03:41:40.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55050 Milliseconds Vbucket 117 2016-04-07T03:41:40.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55080 Milliseconds Vbucket 117 2016-04-07T03:41:40.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55110 Milliseconds Vbucket 117 2016-04-07T03:41:40.825-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3730000 QueuedCount 50000 2016-04-07T03:41:40.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55140 Milliseconds Vbucket 117 2016-04-07T03:41:40.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55170 Milliseconds Vbucket 117 2016-04-07T03:41:40.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55200 Milliseconds Vbucket 117 2016-04-07T03:41:40.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55230 Milliseconds Vbucket 117 2016-04-07T03:41:40.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55260 Milliseconds Vbucket 117 2016-04-07T03:41:40.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55290 Milliseconds Vbucket 117 2016-04-07T03:41:41.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55320 Milliseconds Vbucket 117 2016-04-07T03:41:41.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55350 Milliseconds Vbucket 117 2016-04-07T03:41:41.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55380 Milliseconds Vbucket 117 2016-04-07T03:41:41.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55410 Milliseconds Vbucket 117 2016-04-07T03:41:41.128-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:41.13-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3740000 QueuedCount 49807 2016-04-07T03:41:41.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55440 Milliseconds Vbucket 117 2016-04-07T03:41:41.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55470 Milliseconds Vbucket 117 2016-04-07T03:41:41.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55500 Milliseconds Vbucket 117 2016-04-07T03:41:41.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55530 Milliseconds Vbucket 117 2016-04-07T03:41:41.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55560 Milliseconds Vbucket 117 2016-04-07T03:41:41.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55590 Milliseconds Vbucket 117 2016-04-07T03:41:41.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55620 Milliseconds Vbucket 117 2016-04-07T03:41:41.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55650 Milliseconds Vbucket 117 2016-04-07T03:41:41.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55680 Milliseconds Vbucket 117 2016-04-07T03:41:41.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55710 Milliseconds Vbucket 117 2016-04-07T03:41:41.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55740 Milliseconds Vbucket 117 2016-04-07T03:41:41.436-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3750000 QueuedCount 50000 2016-04-07T03:41:41.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55770 Milliseconds Vbucket 117 2016-04-07T03:41:41.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55800 Milliseconds Vbucket 117 2016-04-07T03:41:41.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55830 Milliseconds Vbucket 117 2016-04-07T03:41:41.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55860 Milliseconds Vbucket 117 2016-04-07T03:41:41.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55890 Milliseconds Vbucket 117 2016-04-07T03:41:41.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55920 Milliseconds Vbucket 117 2016-04-07T03:41:41.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55950 Milliseconds Vbucket 117 2016-04-07T03:41:41.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55980 Milliseconds Vbucket 117 2016-04-07T03:41:41.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56010 Milliseconds Vbucket 117 2016-04-07T03:41:41.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56040 Milliseconds Vbucket 117 2016-04-07T03:41:41.742-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3760000 QueuedCount 50000 2016-04-07T03:41:41.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56070 Milliseconds Vbucket 117 2016-04-07T03:41:41.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56100 Milliseconds Vbucket 117 2016-04-07T03:41:41.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56130 Milliseconds Vbucket 117 2016-04-07T03:41:41.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56160 Milliseconds Vbucket 117 2016-04-07T03:41:41.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56190 Milliseconds Vbucket 117 2016-04-07T03:41:41.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56220 Milliseconds Vbucket 117 2016-04-07T03:41:41.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56250 Milliseconds Vbucket 117 2016-04-07T03:41:41.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56280 Milliseconds Vbucket 117 2016-04-07T03:41:42.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56310 Milliseconds Vbucket 117 2016-04-07T03:41:42.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56340 Milliseconds Vbucket 117 2016-04-07T03:41:42.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3770000 QueuedCount 50000 2016-04-07T03:41:42.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56370 Milliseconds Vbucket 117 2016-04-07T03:41:42.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56400 Milliseconds Vbucket 117 2016-04-07T03:41:42.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56430 Milliseconds Vbucket 117 2016-04-07T03:41:42.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56460 Milliseconds Vbucket 117 2016-04-07T03:41:42.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56490 Milliseconds Vbucket 117 2016-04-07T03:41:42.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56520 Milliseconds Vbucket 117 2016-04-07T03:41:42.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56550 Milliseconds Vbucket 117 2016-04-07T03:41:42.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56580 Milliseconds Vbucket 117 2016-04-07T03:41:42.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56610 Milliseconds Vbucket 117 2016-04-07T03:41:42.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56640 Milliseconds Vbucket 117 2016-04-07T03:41:42.358-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3780000 QueuedCount 47824 2016-04-07T03:41:42.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56670 Milliseconds Vbucket 117 2016-04-07T03:41:42.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56700 Milliseconds Vbucket 117 2016-04-07T03:41:42.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56730 Milliseconds Vbucket 117 2016-04-07T03:41:42.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56760 Milliseconds Vbucket 117 2016-04-07T03:41:42.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56790 Milliseconds Vbucket 117 2016-04-07T03:41:42.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56820 Milliseconds Vbucket 117 2016-04-07T03:41:42.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56850 Milliseconds Vbucket 117 2016-04-07T03:41:42.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56880 Milliseconds Vbucket 117 2016-04-07T03:41:42.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56910 Milliseconds Vbucket 117 2016-04-07T03:41:42.612-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3790000 QueuedCount 37824 2016-04-07T03:41:42.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56940 Milliseconds Vbucket 117 2016-04-07T03:41:42.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56970 Milliseconds Vbucket 117 2016-04-07T03:41:42.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57000 Milliseconds Vbucket 117 2016-04-07T03:41:42.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57030 Milliseconds Vbucket 117 2016-04-07T03:41:42.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57060 Milliseconds Vbucket 117 2016-04-07T03:41:42.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57090 Milliseconds Vbucket 117 2016-04-07T03:41:42.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57120 Milliseconds Vbucket 117 2016-04-07T03:41:42.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57150 Milliseconds Vbucket 117 2016-04-07T03:41:42.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57180 Milliseconds Vbucket 117 2016-04-07T03:41:42.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57210 Milliseconds Vbucket 117 2016-04-07T03:41:42.904-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3800000 QueuedCount 27824 2016-04-07T03:41:42.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57240 Milliseconds Vbucket 117 2016-04-07T03:41:42.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57270 Milliseconds Vbucket 117 2016-04-07T03:41:42.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57300 Milliseconds Vbucket 117 2016-04-07T03:41:43.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57330 Milliseconds Vbucket 117 2016-04-07T03:41:43.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57360 Milliseconds Vbucket 117 2016-04-07T03:41:43.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57390 Milliseconds Vbucket 117 2016-04-07T03:41:43.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57420 Milliseconds Vbucket 117 2016-04-07T03:41:43.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57450 Milliseconds Vbucket 117 2016-04-07T03:41:43.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57480 Milliseconds Vbucket 117 2016-04-07T03:41:43.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57510 Milliseconds Vbucket 117 2016-04-07T03:41:43.217-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3810000 QueuedCount 17824 2016-04-07T03:41:43.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57540 Milliseconds Vbucket 117 2016-04-07T03:41:43.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57570 Milliseconds Vbucket 117 2016-04-07T03:41:43.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57600 Milliseconds Vbucket 117 2016-04-07T03:41:43.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57630 Milliseconds Vbucket 117 2016-04-07T03:41:43.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57660 Milliseconds Vbucket 117 2016-04-07T03:41:43.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57690 Milliseconds Vbucket 117 2016-04-07T03:41:43.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57720 Milliseconds Vbucket 117 2016-04-07T03:41:43.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57750 Milliseconds Vbucket 117 2016-04-07T03:41:43.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57780 Milliseconds Vbucket 117 2016-04-07T03:41:43.483-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3820000 QueuedCount 7824 2016-04-07T03:41:43.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57810 Milliseconds Vbucket 117 2016-04-07T03:41:43.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57840 Milliseconds Vbucket 117 2016-04-07T03:41:43.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57870 Milliseconds Vbucket 117 2016-04-07T03:41:43.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57900 Milliseconds Vbucket 117 2016-04-07T03:41:43.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57930 Milliseconds Vbucket 117 2016-04-07T03:41:43.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57960 Milliseconds Vbucket 117 2016-04-07T03:41:43.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57990 Milliseconds Vbucket 117 2016-04-07T03:41:43.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58020 Milliseconds Vbucket 117 2016-04-07T03:41:43.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58050 Milliseconds Vbucket 117 2016-04-07T03:41:43.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58080 Milliseconds Vbucket 117 2016-04-07T03:41:43.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58110 Milliseconds Vbucket 117 2016-04-07T03:41:43.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58140 Milliseconds Vbucket 117 2016-04-07T03:41:43.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58170 Milliseconds Vbucket 117 2016-04-07T03:41:43.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58200 Milliseconds Vbucket 117 2016-04-07T03:41:43.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58230 Milliseconds Vbucket 117 2016-04-07T03:41:43.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58260 Milliseconds Vbucket 117 2016-04-07T03:41:43.961-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.291µs CommitTime 240.862392ms TotalFlushTime 2m2.664462955s TotalCommitTime 4.511633158s 2016-04-07T03:41:43.961-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 3827824, 3827824, 22 committed:true 2016-04-07T03:41:43.961-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 3827824, 3827824, 22 committed:true) SnapCreateDur 252.640919ms SnapOpenDur 39.144µs 2016-04-07T03:41:43.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58290 Milliseconds Vbucket 117 2016-04-07T03:41:44.006-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:44.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58320 Milliseconds Vbucket 117 2016-04-07T03:41:44.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58350 Milliseconds Vbucket 117 2016-04-07T03:41:44.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58380 Milliseconds Vbucket 117 2016-04-07T03:41:44.075-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3830000 QueuedCount 50000 2016-04-07T03:41:44.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58410 Milliseconds Vbucket 117 2016-04-07T03:41:44.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58440 Milliseconds Vbucket 117 2016-04-07T03:41:44.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58470 Milliseconds Vbucket 117 2016-04-07T03:41:44.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58500 Milliseconds Vbucket 117 2016-04-07T03:41:44.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58530 Milliseconds Vbucket 117 2016-04-07T03:41:44.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58560 Milliseconds Vbucket 117 2016-04-07T03:41:44.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58590 Milliseconds Vbucket 117 2016-04-07T03:41:44.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58620 Milliseconds Vbucket 117 2016-04-07T03:41:44.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58650 Milliseconds Vbucket 117 2016-04-07T03:41:44.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58680 Milliseconds Vbucket 117 2016-04-07T03:41:44.387-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3840000 QueuedCount 50000 2016-04-07T03:41:44.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58710 Milliseconds Vbucket 117 2016-04-07T03:41:44.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58740 Milliseconds Vbucket 117 2016-04-07T03:41:44.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58770 Milliseconds Vbucket 117 2016-04-07T03:41:44.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58800 Milliseconds Vbucket 117 2016-04-07T03:41:44.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58830 Milliseconds Vbucket 117 2016-04-07T03:41:44.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58860 Milliseconds Vbucket 117 2016-04-07T03:41:44.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58890 Milliseconds Vbucket 117 2016-04-07T03:41:44.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58920 Milliseconds Vbucket 117 2016-04-07T03:41:44.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58950 Milliseconds Vbucket 117 2016-04-07T03:41:44.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58980 Milliseconds Vbucket 117 2016-04-07T03:41:44.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59010 Milliseconds Vbucket 117 2016-04-07T03:41:44.719-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3850000 QueuedCount 50000 2016-04-07T03:41:44.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59040 Milliseconds Vbucket 117 2016-04-07T03:41:44.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59070 Milliseconds Vbucket 117 2016-04-07T03:41:44.789-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:44.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59100 Milliseconds Vbucket 117 2016-04-07T03:41:44.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59130 Milliseconds Vbucket 117 2016-04-07T03:41:44.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59160 Milliseconds Vbucket 117 2016-04-07T03:41:44.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59190 Milliseconds Vbucket 117 2016-04-07T03:41:44.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59220 Milliseconds Vbucket 117 2016-04-07T03:41:44.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59250 Milliseconds Vbucket 117 2016-04-07T03:41:44.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59280 Milliseconds Vbucket 117 2016-04-07T03:41:45.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59310 Milliseconds Vbucket 117 2016-04-07T03:41:45.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59340 Milliseconds Vbucket 117 2016-04-07T03:41:45.05-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3860000 QueuedCount 50000 2016-04-07T03:41:45.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59370 Milliseconds Vbucket 117 2016-04-07T03:41:45.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59400 Milliseconds Vbucket 117 2016-04-07T03:41:45.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59430 Milliseconds Vbucket 117 2016-04-07T03:41:45.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59460 Milliseconds Vbucket 117 2016-04-07T03:41:45.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59490 Milliseconds Vbucket 117 2016-04-07T03:41:45.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59520 Milliseconds Vbucket 117 2016-04-07T03:41:45.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59550 Milliseconds Vbucket 117 2016-04-07T03:41:45.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59580 Milliseconds Vbucket 117 2016-04-07T03:41:45.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59610 Milliseconds Vbucket 117 2016-04-07T03:41:45.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59640 Milliseconds Vbucket 117 2016-04-07T03:41:45.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59670 Milliseconds Vbucket 117 2016-04-07T03:41:45.383-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3870000 QueuedCount 50000 2016-04-07T03:41:45.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59700 Milliseconds Vbucket 117 2016-04-07T03:41:45.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59730 Milliseconds Vbucket 117 2016-04-07T03:41:45.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59760 Milliseconds Vbucket 117 2016-04-07T03:41:45.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59790 Milliseconds Vbucket 117 2016-04-07T03:41:45.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59820 Milliseconds Vbucket 117 2016-04-07T03:41:45.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59850 Milliseconds Vbucket 117 2016-04-07T03:41:45.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59880 Milliseconds Vbucket 117 2016-04-07T03:41:45.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59910 Milliseconds Vbucket 117 2016-04-07T03:41:45.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59940 Milliseconds Vbucket 117 2016-04-07T03:41:45.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59970 Milliseconds Vbucket 117 2016-04-07T03:41:45.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60000 Milliseconds Vbucket 117 2016-04-07T03:41:45.702-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3880000 QueuedCount 50000 2016-04-07T03:41:45.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60030 Milliseconds Vbucket 117 2016-04-07T03:41:45.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60060 Milliseconds Vbucket 117 2016-04-07T03:41:45.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60090 Milliseconds Vbucket 117 2016-04-07T03:41:45.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60120 Milliseconds Vbucket 117 2016-04-07T03:41:45.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60150 Milliseconds Vbucket 117 2016-04-07T03:41:45.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60180 Milliseconds Vbucket 117 2016-04-07T03:41:45.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60210 Milliseconds Vbucket 117 2016-04-07T03:41:45.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60240 Milliseconds Vbucket 117 2016-04-07T03:41:45.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60270 Milliseconds Vbucket 117 2016-04-07T03:41:45.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60300 Milliseconds Vbucket 117 2016-04-07T03:41:46.006-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:46.008-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3890000 QueuedCount 49742 2016-04-07T03:41:46.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60330 Milliseconds Vbucket 117 2016-04-07T03:41:46.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60360 Milliseconds Vbucket 117 2016-04-07T03:41:46.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60390 Milliseconds Vbucket 117 2016-04-07T03:41:46.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60420 Milliseconds Vbucket 117 2016-04-07T03:41:46.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60450 Milliseconds Vbucket 117 2016-04-07T03:41:46.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60480 Milliseconds Vbucket 117 2016-04-07T03:41:46.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60510 Milliseconds Vbucket 117 2016-04-07T03:41:46.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60540 Milliseconds Vbucket 117 2016-04-07T03:41:46.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60570 Milliseconds Vbucket 117 2016-04-07T03:41:46.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60600 Milliseconds Vbucket 117 2016-04-07T03:41:46.314-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3900000 QueuedCount 50000 2016-04-07T03:41:46.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60630 Milliseconds Vbucket 117 2016-04-07T03:41:46.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60660 Milliseconds Vbucket 117 2016-04-07T03:41:46.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60690 Milliseconds Vbucket 117 2016-04-07T03:41:46.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60720 Milliseconds Vbucket 117 2016-04-07T03:41:46.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60750 Milliseconds Vbucket 117 2016-04-07T03:41:46.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60780 Milliseconds Vbucket 117 2016-04-07T03:41:46.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60810 Milliseconds Vbucket 117 2016-04-07T03:41:46.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60840 Milliseconds Vbucket 117 2016-04-07T03:41:46.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60870 Milliseconds Vbucket 117 2016-04-07T03:41:46.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60900 Milliseconds Vbucket 117 2016-04-07T03:41:46.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60930 Milliseconds Vbucket 117 2016-04-07T03:41:46.639-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3910000 QueuedCount 50000 2016-04-07T03:41:46.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60960 Milliseconds Vbucket 117 2016-04-07T03:41:46.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60990 Milliseconds Vbucket 117 2016-04-07T03:41:46.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61020 Milliseconds Vbucket 117 2016-04-07T03:41:46.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61050 Milliseconds Vbucket 117 2016-04-07T03:41:46.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61080 Milliseconds Vbucket 117 2016-04-07T03:41:46.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61110 Milliseconds Vbucket 117 2016-04-07T03:41:46.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61140 Milliseconds Vbucket 117 2016-04-07T03:41:46.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61170 Milliseconds Vbucket 117 2016-04-07T03:41:46.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61200 Milliseconds Vbucket 117 2016-04-07T03:41:46.919-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3920000 QueuedCount 50000 2016-04-07T03:41:46.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61230 Milliseconds Vbucket 117 2016-04-07T03:41:46.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61260 Milliseconds Vbucket 117 2016-04-07T03:41:46.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61290 Milliseconds Vbucket 117 2016-04-07T03:41:47.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61320 Milliseconds Vbucket 117 2016-04-07T03:41:47.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61350 Milliseconds Vbucket 117 2016-04-07T03:41:47.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61380 Milliseconds Vbucket 117 2016-04-07T03:41:47.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61410 Milliseconds Vbucket 117 2016-04-07T03:41:47.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61440 Milliseconds Vbucket 117 2016-04-07T03:41:47.146-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:47.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61470 Milliseconds Vbucket 117 2016-04-07T03:41:47.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61500 Milliseconds Vbucket 117 2016-04-07T03:41:47.216-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3930000 QueuedCount 50000 2016-04-07T03:41:47.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61530 Milliseconds Vbucket 117 2016-04-07T03:41:47.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61560 Milliseconds Vbucket 117 2016-04-07T03:41:47.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61590 Milliseconds Vbucket 117 2016-04-07T03:41:47.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61620 Milliseconds Vbucket 117 2016-04-07T03:41:47.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61650 Milliseconds Vbucket 117 2016-04-07T03:41:47.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61680 Milliseconds Vbucket 117 2016-04-07T03:41:47.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61710 Milliseconds Vbucket 117 2016-04-07T03:41:47.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61740 Milliseconds Vbucket 117 2016-04-07T03:41:47.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61770 Milliseconds Vbucket 117 2016-04-07T03:41:47.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61800 Milliseconds Vbucket 117 2016-04-07T03:41:47.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61830 Milliseconds Vbucket 117 2016-04-07T03:41:47.533-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3940000 QueuedCount 50000 2016-04-07T03:41:47.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61860 Milliseconds Vbucket 117 2016-04-07T03:41:47.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61890 Milliseconds Vbucket 117 2016-04-07T03:41:47.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61920 Milliseconds Vbucket 117 2016-04-07T03:41:47.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61950 Milliseconds Vbucket 117 2016-04-07T03:41:47.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61980 Milliseconds Vbucket 117 2016-04-07T03:41:47.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62010 Milliseconds Vbucket 117 2016-04-07T03:41:47.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62040 Milliseconds Vbucket 117 2016-04-07T03:41:47.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62070 Milliseconds Vbucket 117 2016-04-07T03:41:47.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62100 Milliseconds Vbucket 117 2016-04-07T03:41:47.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62130 Milliseconds Vbucket 117 2016-04-07T03:41:47.847-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3950000 QueuedCount 50000 2016-04-07T03:41:47.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62160 Milliseconds Vbucket 117 2016-04-07T03:41:47.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62190 Milliseconds Vbucket 117 2016-04-07T03:41:47.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62220 Milliseconds Vbucket 117 2016-04-07T03:41:47.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62250 Milliseconds Vbucket 117 2016-04-07T03:41:47.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62280 Milliseconds Vbucket 117 2016-04-07T03:41:48.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62310 Milliseconds Vbucket 117 2016-04-07T03:41:48.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62340 Milliseconds Vbucket 117 2016-04-07T03:41:48.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62370 Milliseconds Vbucket 117 2016-04-07T03:41:48.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62400 Milliseconds Vbucket 117 2016-04-07T03:41:48.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62430 Milliseconds Vbucket 117 2016-04-07T03:41:48.143-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3960000 QueuedCount 50000 2016-04-07T03:41:48.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62460 Milliseconds Vbucket 117 2016-04-07T03:41:48.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62490 Milliseconds Vbucket 117 2016-04-07T03:41:48.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62520 Milliseconds Vbucket 117 2016-04-07T03:41:48.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62550 Milliseconds Vbucket 117 2016-04-07T03:41:48.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62580 Milliseconds Vbucket 117 2016-04-07T03:41:48.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62610 Milliseconds Vbucket 117 2016-04-07T03:41:48.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62640 Milliseconds Vbucket 117 2016-04-07T03:41:48.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62670 Milliseconds Vbucket 117 2016-04-07T03:41:48.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62700 Milliseconds Vbucket 117 2016-04-07T03:41:48.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62730 Milliseconds Vbucket 117 2016-04-07T03:41:48.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62760 Milliseconds Vbucket 117 2016-04-07T03:41:48.46-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3970000 QueuedCount 44706 2016-04-07T03:41:48.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62790 Milliseconds Vbucket 117 2016-04-07T03:41:48.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62820 Milliseconds Vbucket 117 2016-04-07T03:41:48.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62850 Milliseconds Vbucket 117 2016-04-07T03:41:48.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62880 Milliseconds Vbucket 117 2016-04-07T03:41:48.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62910 Milliseconds Vbucket 117 2016-04-07T03:41:48.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62940 Milliseconds Vbucket 117 2016-04-07T03:41:48.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62970 Milliseconds Vbucket 117 2016-04-07T03:41:48.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63000 Milliseconds Vbucket 117 2016-04-07T03:41:48.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63030 Milliseconds Vbucket 117 2016-04-07T03:41:48.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63060 Milliseconds Vbucket 117 2016-04-07T03:41:48.764-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3980000 QueuedCount 34706 2016-04-07T03:41:48.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63090 Milliseconds Vbucket 117 2016-04-07T03:41:48.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63120 Milliseconds Vbucket 117 2016-04-07T03:41:48.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63150 Milliseconds Vbucket 117 2016-04-07T03:41:48.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63180 Milliseconds Vbucket 117 2016-04-07T03:41:48.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63210 Milliseconds Vbucket 117 2016-04-07T03:41:48.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63240 Milliseconds Vbucket 117 2016-04-07T03:41:48.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63270 Milliseconds Vbucket 117 2016-04-07T03:41:48.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63300 Milliseconds Vbucket 117 2016-04-07T03:41:49.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63330 Milliseconds Vbucket 117 2016-04-07T03:41:49.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63360 Milliseconds Vbucket 117 2016-04-07T03:41:49.082-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 3990000 QueuedCount 24706 2016-04-07T03:41:49.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63390 Milliseconds Vbucket 117 2016-04-07T03:41:49.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63420 Milliseconds Vbucket 117 2016-04-07T03:41:49.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63450 Milliseconds Vbucket 117 2016-04-07T03:41:49.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63480 Milliseconds Vbucket 117 2016-04-07T03:41:49.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63510 Milliseconds Vbucket 117 2016-04-07T03:41:49.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63540 Milliseconds Vbucket 117 2016-04-07T03:41:49.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63570 Milliseconds Vbucket 117 2016-04-07T03:41:49.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63600 Milliseconds Vbucket 117 2016-04-07T03:41:49.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63630 Milliseconds Vbucket 117 2016-04-07T03:41:49.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63660 Milliseconds Vbucket 117 2016-04-07T03:41:49.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63690 Milliseconds Vbucket 117 2016-04-07T03:41:49.399-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4000000 QueuedCount 14706 2016-04-07T03:41:49.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63720 Milliseconds Vbucket 117 2016-04-07T03:41:49.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63750 Milliseconds Vbucket 117 2016-04-07T03:41:49.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63780 Milliseconds Vbucket 117 2016-04-07T03:41:49.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63810 Milliseconds Vbucket 117 2016-04-07T03:41:49.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63840 Milliseconds Vbucket 117 2016-04-07T03:41:49.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63870 Milliseconds Vbucket 117 2016-04-07T03:41:49.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63900 Milliseconds Vbucket 117 2016-04-07T03:41:49.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63930 Milliseconds Vbucket 117 2016-04-07T03:41:49.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63960 Milliseconds Vbucket 117 2016-04-07T03:41:49.677-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4010000 QueuedCount 4706 2016-04-07T03:41:49.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63990 Milliseconds Vbucket 117 2016-04-07T03:41:49.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64020 Milliseconds Vbucket 117 2016-04-07T03:41:49.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64050 Milliseconds Vbucket 117 2016-04-07T03:41:49.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64080 Milliseconds Vbucket 117 2016-04-07T03:41:49.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64110 Milliseconds Vbucket 117 2016-04-07T03:41:49.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64140 Milliseconds Vbucket 117 2016-04-07T03:41:49.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64170 Milliseconds Vbucket 117 2016-04-07T03:41:49.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64200 Milliseconds Vbucket 117 2016-04-07T03:41:49.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64230 Milliseconds Vbucket 117 2016-04-07T03:41:49.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64260 Milliseconds Vbucket 117 2016-04-07T03:41:49.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64290 Milliseconds Vbucket 117 2016-04-07T03:41:50.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64320 Milliseconds Vbucket 117 2016-04-07T03:41:50.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64350 Milliseconds Vbucket 117 2016-04-07T03:41:50.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64380 Milliseconds Vbucket 117 2016-04-07T03:41:50.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64410 Milliseconds Vbucket 117 2016-04-07T03:41:50.114-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.979µs CommitTime 246.503453ms TotalFlushTime 2m8.233202082s TotalCommitTime 4.758136611s 2016-04-07T03:41:50.114-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 4014706, 4014706, 23 committed:true 2016-04-07T03:41:50.114-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 4014706, 4014706, 23 committed:true) SnapCreateDur 258.018962ms SnapOpenDur 42.554µs 2016-04-07T03:41:50.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64440 Milliseconds Vbucket 117 2016-04-07T03:41:50.161-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:50.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64470 Milliseconds Vbucket 117 2016-04-07T03:41:50.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64500 Milliseconds Vbucket 117 2016-04-07T03:41:50.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64530 Milliseconds Vbucket 117 2016-04-07T03:41:50.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64560 Milliseconds Vbucket 117 2016-04-07T03:41:50.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64590 Milliseconds Vbucket 117 2016-04-07T03:41:50.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64620 Milliseconds Vbucket 117 2016-04-07T03:41:50.32-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4020000 QueuedCount 50000 2016-04-07T03:41:50.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64650 Milliseconds Vbucket 117 2016-04-07T03:41:50.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64680 Milliseconds Vbucket 117 2016-04-07T03:41:50.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64710 Milliseconds Vbucket 117 2016-04-07T03:41:50.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64740 Milliseconds Vbucket 117 2016-04-07T03:41:50.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64770 Milliseconds Vbucket 117 2016-04-07T03:41:50.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64800 Milliseconds Vbucket 117 2016-04-07T03:41:50.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64830 Milliseconds Vbucket 117 2016-04-07T03:41:50.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64860 Milliseconds Vbucket 117 2016-04-07T03:41:50.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64890 Milliseconds Vbucket 117 2016-04-07T03:41:50.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64920 Milliseconds Vbucket 117 2016-04-07T03:41:50.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64950 Milliseconds Vbucket 117 2016-04-07T03:41:50.669-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4030000 QueuedCount 50000 2016-04-07T03:41:50.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64980 Milliseconds Vbucket 117 2016-04-07T03:41:50.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65010 Milliseconds Vbucket 117 2016-04-07T03:41:50.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65040 Milliseconds Vbucket 117 2016-04-07T03:41:50.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65070 Milliseconds Vbucket 117 2016-04-07T03:41:50.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65100 Milliseconds Vbucket 117 2016-04-07T03:41:50.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65130 Milliseconds Vbucket 117 2016-04-07T03:41:50.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65160 Milliseconds Vbucket 117 2016-04-07T03:41:50.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65190 Milliseconds Vbucket 117 2016-04-07T03:41:50.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65220 Milliseconds Vbucket 117 2016-04-07T03:41:50.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65250 Milliseconds Vbucket 117 2016-04-07T03:41:50.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65280 Milliseconds Vbucket 117 2016-04-07T03:41:51.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65310 Milliseconds Vbucket 117 2016-04-07T03:41:51.01-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:51.015-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4040000 QueuedCount 49478 2016-04-07T03:41:51.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65340 Milliseconds Vbucket 117 2016-04-07T03:41:51.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65370 Milliseconds Vbucket 117 2016-04-07T03:41:51.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65400 Milliseconds Vbucket 117 2016-04-07T03:41:51.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65430 Milliseconds Vbucket 117 2016-04-07T03:41:51.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65460 Milliseconds Vbucket 117 2016-04-07T03:41:51.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65490 Milliseconds Vbucket 117 2016-04-07T03:41:51.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65520 Milliseconds Vbucket 117 2016-04-07T03:41:51.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65550 Milliseconds Vbucket 117 2016-04-07T03:41:51.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65580 Milliseconds Vbucket 117 2016-04-07T03:41:51.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65610 Milliseconds Vbucket 117 2016-04-07T03:41:51.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65640 Milliseconds Vbucket 117 2016-04-07T03:41:51.355-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4050000 QueuedCount 50000 2016-04-07T03:41:51.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65670 Milliseconds Vbucket 117 2016-04-07T03:41:51.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65700 Milliseconds Vbucket 117 2016-04-07T03:41:51.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65730 Milliseconds Vbucket 117 2016-04-07T03:41:51.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65760 Milliseconds Vbucket 117 2016-04-07T03:41:51.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65790 Milliseconds Vbucket 117 2016-04-07T03:41:51.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65820 Milliseconds Vbucket 117 2016-04-07T03:41:51.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65850 Milliseconds Vbucket 117 2016-04-07T03:41:51.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65880 Milliseconds Vbucket 117 2016-04-07T03:41:51.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65910 Milliseconds Vbucket 117 2016-04-07T03:41:51.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65940 Milliseconds Vbucket 117 2016-04-07T03:41:51.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65970 Milliseconds Vbucket 117 2016-04-07T03:41:51.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66000 Milliseconds Vbucket 117 2016-04-07T03:41:51.701-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4060000 QueuedCount 50000 2016-04-07T03:41:51.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66030 Milliseconds Vbucket 117 2016-04-07T03:41:51.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66060 Milliseconds Vbucket 117 2016-04-07T03:41:51.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66090 Milliseconds Vbucket 117 2016-04-07T03:41:51.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66120 Milliseconds Vbucket 117 2016-04-07T03:41:51.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66150 Milliseconds Vbucket 117 2016-04-07T03:41:51.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66180 Milliseconds Vbucket 117 2016-04-07T03:41:51.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66210 Milliseconds Vbucket 117 2016-04-07T03:41:51.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66240 Milliseconds Vbucket 117 2016-04-07T03:41:51.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66270 Milliseconds Vbucket 117 2016-04-07T03:41:51.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66300 Milliseconds Vbucket 117 2016-04-07T03:41:52.005-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4070000 QueuedCount 50000 2016-04-07T03:41:52.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66330 Milliseconds Vbucket 117 2016-04-07T03:41:52.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66360 Milliseconds Vbucket 117 2016-04-07T03:41:52.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66390 Milliseconds Vbucket 117 2016-04-07T03:41:52.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66420 Milliseconds Vbucket 117 2016-04-07T03:41:52.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66450 Milliseconds Vbucket 117 2016-04-07T03:41:52.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66480 Milliseconds Vbucket 117 2016-04-07T03:41:52.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66510 Milliseconds Vbucket 117 2016-04-07T03:41:52.205-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:52.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66540 Milliseconds Vbucket 117 2016-04-07T03:41:52.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66570 Milliseconds Vbucket 117 2016-04-07T03:41:52.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66600 Milliseconds Vbucket 117 2016-04-07T03:41:52.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66630 Milliseconds Vbucket 117 2016-04-07T03:41:52.328-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4080000 QueuedCount 50000 2016-04-07T03:41:52.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66660 Milliseconds Vbucket 117 2016-04-07T03:41:52.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66690 Milliseconds Vbucket 117 2016-04-07T03:41:52.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66720 Milliseconds Vbucket 117 2016-04-07T03:41:52.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66750 Milliseconds Vbucket 117 2016-04-07T03:41:52.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66780 Milliseconds Vbucket 117 2016-04-07T03:41:52.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66810 Milliseconds Vbucket 117 2016-04-07T03:41:52.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66840 Milliseconds Vbucket 117 2016-04-07T03:41:52.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66870 Milliseconds Vbucket 117 2016-04-07T03:41:52.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66900 Milliseconds Vbucket 117 2016-04-07T03:41:52.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66930 Milliseconds Vbucket 117 2016-04-07T03:41:52.641-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4090000 QueuedCount 50000 2016-04-07T03:41:52.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66960 Milliseconds Vbucket 117 2016-04-07T03:41:52.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66990 Milliseconds Vbucket 117 2016-04-07T03:41:52.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67020 Milliseconds Vbucket 117 2016-04-07T03:41:52.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67050 Milliseconds Vbucket 117 2016-04-07T03:41:52.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67080 Milliseconds Vbucket 117 2016-04-07T03:41:52.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67110 Milliseconds Vbucket 117 2016-04-07T03:41:52.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67140 Milliseconds Vbucket 117 2016-04-07T03:41:52.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67170 Milliseconds Vbucket 117 2016-04-07T03:41:52.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67200 Milliseconds Vbucket 117 2016-04-07T03:41:52.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67230 Milliseconds Vbucket 117 2016-04-07T03:41:52.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67260 Milliseconds Vbucket 117 2016-04-07T03:41:52.971-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4100000 QueuedCount 50000 2016-04-07T03:41:52.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67290 Milliseconds Vbucket 117 2016-04-07T03:41:53.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67320 Milliseconds Vbucket 117 2016-04-07T03:41:53.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67350 Milliseconds Vbucket 117 2016-04-07T03:41:53.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67380 Milliseconds Vbucket 117 2016-04-07T03:41:53.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67410 Milliseconds Vbucket 117 2016-04-07T03:41:53.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67440 Milliseconds Vbucket 117 2016-04-07T03:41:53.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67470 Milliseconds Vbucket 117 2016-04-07T03:41:53.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67500 Milliseconds Vbucket 117 2016-04-07T03:41:53.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67530 Milliseconds Vbucket 117 2016-04-07T03:41:53.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67560 Milliseconds Vbucket 117 2016-04-07T03:41:53.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67590 Milliseconds Vbucket 117 2016-04-07T03:41:53.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67620 Milliseconds Vbucket 117 2016-04-07T03:41:53.322-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4110000 QueuedCount 50000 2016-04-07T03:41:53.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67650 Milliseconds Vbucket 117 2016-04-07T03:41:53.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67680 Milliseconds Vbucket 117 2016-04-07T03:41:53.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67710 Milliseconds Vbucket 117 2016-04-07T03:41:53.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67740 Milliseconds Vbucket 117 2016-04-07T03:41:53.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67770 Milliseconds Vbucket 117 2016-04-07T03:41:53.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67800 Milliseconds Vbucket 117 2016-04-07T03:41:53.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67830 Milliseconds Vbucket 117 2016-04-07T03:41:53.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67860 Milliseconds Vbucket 117 2016-04-07T03:41:53.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67890 Milliseconds Vbucket 117 2016-04-07T03:41:53.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67920 Milliseconds Vbucket 117 2016-04-07T03:41:53.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67950 Milliseconds Vbucket 117 2016-04-07T03:41:53.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67980 Milliseconds Vbucket 117 2016-04-07T03:41:53.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68010 Milliseconds Vbucket 117 2016-04-07T03:41:53.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68040 Milliseconds Vbucket 117 2016-04-07T03:41:53.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68070 Milliseconds Vbucket 117 2016-04-07T03:41:53.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68100 Milliseconds Vbucket 117 2016-04-07T03:41:53.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68130 Milliseconds Vbucket 117 2016-04-07T03:41:53.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68160 Milliseconds Vbucket 117 2016-04-07T03:41:53.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68190 Milliseconds Vbucket 117 2016-04-07T03:41:53.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68220 Milliseconds Vbucket 117 2016-04-07T03:41:53.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68250 Milliseconds Vbucket 117 2016-04-07T03:41:53.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68280 Milliseconds Vbucket 117 2016-04-07T03:41:54.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68310 Milliseconds Vbucket 117 2016-04-07T03:41:54.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68340 Milliseconds Vbucket 117 2016-04-07T03:41:54.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68370 Milliseconds Vbucket 117 2016-04-07T03:41:54.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68400 Milliseconds Vbucket 117 2016-04-07T03:41:54.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68430 Milliseconds Vbucket 117 2016-04-07T03:41:54.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68460 Milliseconds Vbucket 117 2016-04-07T03:41:54.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68490 Milliseconds Vbucket 117 2016-04-07T03:41:54.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68520 Milliseconds Vbucket 117 2016-04-07T03:41:54.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68550 Milliseconds Vbucket 117 2016-04-07T03:41:54.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68580 Milliseconds Vbucket 117 2016-04-07T03:41:54.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68610 Milliseconds Vbucket 117 2016-04-07T03:41:54.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68640 Milliseconds Vbucket 117 2016-04-07T03:41:54.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68670 Milliseconds Vbucket 117 2016-04-07T03:41:54.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68700 Milliseconds Vbucket 117 2016-04-07T03:41:54.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68730 Milliseconds Vbucket 117 2016-04-07T03:41:54.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68760 Milliseconds Vbucket 117 2016-04-07T03:41:54.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68790 Milliseconds Vbucket 117 2016-04-07T03:41:54.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68820 Milliseconds Vbucket 117 2016-04-07T03:41:54.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68850 Milliseconds Vbucket 117 2016-04-07T03:41:54.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68880 Milliseconds Vbucket 117 2016-04-07T03:41:54.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68910 Milliseconds Vbucket 117 2016-04-07T03:41:54.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68940 Milliseconds Vbucket 117 2016-04-07T03:41:54.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68970 Milliseconds Vbucket 117 2016-04-07T03:41:54.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69000 Milliseconds Vbucket 117 2016-04-07T03:41:54.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69030 Milliseconds Vbucket 117 2016-04-07T03:41:54.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69060 Milliseconds Vbucket 117 2016-04-07T03:41:54.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69090 Milliseconds Vbucket 117 2016-04-07T03:41:54.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69120 Milliseconds Vbucket 117 2016-04-07T03:41:54.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69150 Milliseconds Vbucket 117 2016-04-07T03:41:54.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69180 Milliseconds Vbucket 117 2016-04-07T03:41:54.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69210 Milliseconds Vbucket 117 2016-04-07T03:41:54.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69240 Milliseconds Vbucket 117 2016-04-07T03:41:54.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69270 Milliseconds Vbucket 117 2016-04-07T03:41:54.981-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4120000 QueuedCount 41076 2016-04-07T03:41:54.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69300 Milliseconds Vbucket 117 2016-04-07T03:41:55.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69330 Milliseconds Vbucket 117 2016-04-07T03:41:55.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69360 Milliseconds Vbucket 117 2016-04-07T03:41:55.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69390 Milliseconds Vbucket 117 2016-04-07T03:41:55.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69420 Milliseconds Vbucket 117 2016-04-07T03:41:55.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69450 Milliseconds Vbucket 117 2016-04-07T03:41:55.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69480 Milliseconds Vbucket 117 2016-04-07T03:41:55.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69510 Milliseconds Vbucket 117 2016-04-07T03:41:55.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69540 Milliseconds Vbucket 117 2016-04-07T03:41:55.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69570 Milliseconds Vbucket 117 2016-04-07T03:41:55.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69600 Milliseconds Vbucket 117 2016-04-07T03:41:55.305-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4130000 QueuedCount 31076 2016-04-07T03:41:55.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69630 Milliseconds Vbucket 117 2016-04-07T03:41:55.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69660 Milliseconds Vbucket 117 2016-04-07T03:41:55.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69690 Milliseconds Vbucket 117 2016-04-07T03:41:55.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69720 Milliseconds Vbucket 117 2016-04-07T03:41:55.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69750 Milliseconds Vbucket 117 2016-04-07T03:41:55.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69780 Milliseconds Vbucket 117 2016-04-07T03:41:55.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69810 Milliseconds Vbucket 117 2016-04-07T03:41:55.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69840 Milliseconds Vbucket 117 2016-04-07T03:41:55.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69870 Milliseconds Vbucket 117 2016-04-07T03:41:55.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69900 Milliseconds Vbucket 117 2016-04-07T03:41:55.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69930 Milliseconds Vbucket 117 2016-04-07T03:41:55.629-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4140000 QueuedCount 21076 2016-04-07T03:41:55.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69960 Milliseconds Vbucket 117 2016-04-07T03:41:55.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69990 Milliseconds Vbucket 117 2016-04-07T03:41:55.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70020 Milliseconds Vbucket 117 2016-04-07T03:41:55.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70050 Milliseconds Vbucket 117 2016-04-07T03:41:55.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70080 Milliseconds Vbucket 117 2016-04-07T03:41:55.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70110 Milliseconds Vbucket 117 2016-04-07T03:41:55.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70140 Milliseconds Vbucket 117 2016-04-07T03:41:55.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70170 Milliseconds Vbucket 117 2016-04-07T03:41:55.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70200 Milliseconds Vbucket 117 2016-04-07T03:41:55.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70230 Milliseconds Vbucket 117 2016-04-07T03:41:55.935-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4150000 QueuedCount 11076 2016-04-07T03:41:55.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70260 Milliseconds Vbucket 117 2016-04-07T03:41:55.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70290 Milliseconds Vbucket 117 2016-04-07T03:41:56.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70320 Milliseconds Vbucket 117 2016-04-07T03:41:56.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70350 Milliseconds Vbucket 117 2016-04-07T03:41:56.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70380 Milliseconds Vbucket 117 2016-04-07T03:41:56.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70410 Milliseconds Vbucket 117 2016-04-07T03:41:56.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70440 Milliseconds Vbucket 117 2016-04-07T03:41:56.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70470 Milliseconds Vbucket 117 2016-04-07T03:41:56.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70500 Milliseconds Vbucket 117 2016-04-07T03:41:56.201-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4160000 QueuedCount 1076 2016-04-07T03:41:56.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70530 Milliseconds Vbucket 117 2016-04-07T03:41:56.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70560 Milliseconds Vbucket 117 2016-04-07T03:41:56.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70590 Milliseconds Vbucket 117 2016-04-07T03:41:56.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70620 Milliseconds Vbucket 117 2016-04-07T03:41:56.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70650 Milliseconds Vbucket 117 2016-04-07T03:41:56.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70680 Milliseconds Vbucket 117 2016-04-07T03:41:56.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70710 Milliseconds Vbucket 117 2016-04-07T03:41:56.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70740 Milliseconds Vbucket 117 2016-04-07T03:41:56.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70770 Milliseconds Vbucket 117 2016-04-07T03:41:56.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70800 Milliseconds Vbucket 117 2016-04-07T03:41:56.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70830 Milliseconds Vbucket 117 2016-04-07T03:41:56.531-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.556µs CommitTime 262.50726ms TotalFlushTime 2m14.131560548s TotalCommitTime 5.020643871s 2016-04-07T03:41:56.531-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 4161076, 4161076, 24 committed:true 2016-04-07T03:41:56.531-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 4161076, 4161076, 24 committed:true) SnapCreateDur 272.679053ms SnapOpenDur 58.275µs 2016-04-07T03:41:56.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70860 Milliseconds Vbucket 117 2016-04-07T03:41:56.578-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:56.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70890 Milliseconds Vbucket 117 2016-04-07T03:41:56.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70920 Milliseconds Vbucket 117 2016-04-07T03:41:56.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70950 Milliseconds Vbucket 117 2016-04-07T03:41:56.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70980 Milliseconds Vbucket 117 2016-04-07T03:41:56.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71010 Milliseconds Vbucket 117 2016-04-07T03:41:56.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71040 Milliseconds Vbucket 117 2016-04-07T03:41:56.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71070 Milliseconds Vbucket 117 2016-04-07T03:41:56.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71100 Milliseconds Vbucket 117 2016-04-07T03:41:56.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71130 Milliseconds Vbucket 117 2016-04-07T03:41:56.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71160 Milliseconds Vbucket 117 2016-04-07T03:41:56.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71190 Milliseconds Vbucket 117 2016-04-07T03:41:56.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71220 Milliseconds Vbucket 117 2016-04-07T03:41:56.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71250 Milliseconds Vbucket 117 2016-04-07T03:41:56.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71280 Milliseconds Vbucket 117 2016-04-07T03:41:57.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71310 Milliseconds Vbucket 117 2016-04-07T03:41:57.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71340 Milliseconds Vbucket 117 2016-04-07T03:41:57.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71370 Milliseconds Vbucket 117 2016-04-07T03:41:57.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71400 Milliseconds Vbucket 117 2016-04-07T03:41:57.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71430 Milliseconds Vbucket 117 2016-04-07T03:41:57.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71460 Milliseconds Vbucket 117 2016-04-07T03:41:57.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71490 Milliseconds Vbucket 117 2016-04-07T03:41:57.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71520 Milliseconds Vbucket 117 2016-04-07T03:41:57.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71550 Milliseconds Vbucket 117 2016-04-07T03:41:57.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71580 Milliseconds Vbucket 117 2016-04-07T03:41:57.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71610 Milliseconds Vbucket 117 2016-04-07T03:41:57.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71640 Milliseconds Vbucket 117 2016-04-07T03:41:57.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71670 Milliseconds Vbucket 117 2016-04-07T03:41:57.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71700 Milliseconds Vbucket 117 2016-04-07T03:41:57.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71730 Milliseconds Vbucket 117 2016-04-07T03:41:57.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71760 Milliseconds Vbucket 117 2016-04-07T03:41:57.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71790 Milliseconds Vbucket 117 2016-04-07T03:41:57.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71820 Milliseconds Vbucket 117 2016-04-07T03:41:57.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71850 Milliseconds Vbucket 117 2016-04-07T03:41:57.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71880 Milliseconds Vbucket 117 2016-04-07T03:41:57.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71910 Milliseconds Vbucket 117 2016-04-07T03:41:57.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71940 Milliseconds Vbucket 117 2016-04-07T03:41:57.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71970 Milliseconds Vbucket 117 2016-04-07T03:41:57.679-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4170000 QueuedCount 50000 2016-04-07T03:41:57.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72000 Milliseconds Vbucket 117 2016-04-07T03:41:57.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72030 Milliseconds Vbucket 117 2016-04-07T03:41:57.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72060 Milliseconds Vbucket 117 2016-04-07T03:41:57.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72090 Milliseconds Vbucket 117 2016-04-07T03:41:57.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72120 Milliseconds Vbucket 117 2016-04-07T03:41:57.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72150 Milliseconds Vbucket 117 2016-04-07T03:41:57.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72180 Milliseconds Vbucket 117 2016-04-07T03:41:57.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72210 Milliseconds Vbucket 117 2016-04-07T03:41:57.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72240 Milliseconds Vbucket 117 2016-04-07T03:41:57.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72270 Milliseconds Vbucket 117 2016-04-07T03:41:57.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72300 Milliseconds Vbucket 117 2016-04-07T03:41:58.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72330 Milliseconds Vbucket 117 2016-04-07T03:41:58.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72360 Milliseconds Vbucket 117 2016-04-07T03:41:58.069-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4180000 QueuedCount 50000 2016-04-07T03:41:58.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72390 Milliseconds Vbucket 117 2016-04-07T03:41:58.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72420 Milliseconds Vbucket 117 2016-04-07T03:41:58.136-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:41:58.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72450 Milliseconds Vbucket 117 2016-04-07T03:41:58.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72480 Milliseconds Vbucket 117 2016-04-07T03:41:58.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72510 Milliseconds Vbucket 117 2016-04-07T03:41:58.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72540 Milliseconds Vbucket 117 2016-04-07T03:41:58.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72570 Milliseconds Vbucket 117 2016-04-07T03:41:58.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72600 Milliseconds Vbucket 117 2016-04-07T03:41:58.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72630 Milliseconds Vbucket 117 2016-04-07T03:41:58.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72660 Milliseconds Vbucket 117 2016-04-07T03:41:58.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72690 Milliseconds Vbucket 117 2016-04-07T03:41:58.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72720 Milliseconds Vbucket 117 2016-04-07T03:41:58.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72750 Milliseconds Vbucket 117 2016-04-07T03:41:58.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72780 Milliseconds Vbucket 117 2016-04-07T03:41:58.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72810 Milliseconds Vbucket 117 2016-04-07T03:41:58.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72840 Milliseconds Vbucket 117 2016-04-07T03:41:58.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72870 Milliseconds Vbucket 117 2016-04-07T03:41:58.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72900 Milliseconds Vbucket 117 2016-04-07T03:41:58.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72930 Milliseconds Vbucket 117 2016-04-07T03:41:58.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72960 Milliseconds Vbucket 117 2016-04-07T03:41:58.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72990 Milliseconds Vbucket 117 2016-04-07T03:41:58.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73020 Milliseconds Vbucket 117 2016-04-07T03:41:58.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73050 Milliseconds Vbucket 117 2016-04-07T03:41:58.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73080 Milliseconds Vbucket 117 2016-04-07T03:41:58.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73110 Milliseconds Vbucket 117 2016-04-07T03:41:58.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73140 Milliseconds Vbucket 117 2016-04-07T03:41:58.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73170 Milliseconds Vbucket 117 2016-04-07T03:41:58.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73200 Milliseconds Vbucket 117 2016-04-07T03:41:58.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73230 Milliseconds Vbucket 117 2016-04-07T03:41:58.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73260 Milliseconds Vbucket 117 2016-04-07T03:41:58.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73290 Milliseconds Vbucket 117 2016-04-07T03:41:59.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73320 Milliseconds Vbucket 117 2016-04-07T03:41:59.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73350 Milliseconds Vbucket 117 2016-04-07T03:41:59.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73380 Milliseconds Vbucket 117 2016-04-07T03:41:59.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73410 Milliseconds Vbucket 117 2016-04-07T03:41:59.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73440 Milliseconds Vbucket 117 2016-04-07T03:41:59.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73470 Milliseconds Vbucket 117 2016-04-07T03:41:59.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73500 Milliseconds Vbucket 117 2016-04-07T03:41:59.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73530 Milliseconds Vbucket 117 2016-04-07T03:41:59.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73560 Milliseconds Vbucket 117 2016-04-07T03:41:59.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73590 Milliseconds Vbucket 117 2016-04-07T03:41:59.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73620 Milliseconds Vbucket 117 2016-04-07T03:41:59.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73650 Milliseconds Vbucket 117 2016-04-07T03:41:59.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73680 Milliseconds Vbucket 117 2016-04-07T03:41:59.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73710 Milliseconds Vbucket 117 2016-04-07T03:41:59.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73740 Milliseconds Vbucket 117 2016-04-07T03:41:59.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73770 Milliseconds Vbucket 117 2016-04-07T03:41:59.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73800 Milliseconds Vbucket 117 2016-04-07T03:41:59.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73830 Milliseconds Vbucket 117 2016-04-07T03:41:59.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73860 Milliseconds Vbucket 117 2016-04-07T03:41:59.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73890 Milliseconds Vbucket 117 2016-04-07T03:41:59.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73920 Milliseconds Vbucket 117 2016-04-07T03:41:59.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73950 Milliseconds Vbucket 117 2016-04-07T03:41:59.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73980 Milliseconds Vbucket 117 2016-04-07T03:41:59.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74010 Milliseconds Vbucket 117 2016-04-07T03:41:59.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74040 Milliseconds Vbucket 117 2016-04-07T03:41:59.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74070 Milliseconds Vbucket 117 2016-04-07T03:41:59.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74100 Milliseconds Vbucket 117 2016-04-07T03:41:59.822-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4190000 QueuedCount 50000 2016-04-07T03:41:59.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74130 Milliseconds Vbucket 117 2016-04-07T03:41:59.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74160 Milliseconds Vbucket 117 2016-04-07T03:41:59.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74190 Milliseconds Vbucket 117 2016-04-07T03:41:59.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74220 Milliseconds Vbucket 117 2016-04-07T03:41:59.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74250 Milliseconds Vbucket 117 2016-04-07T03:41:59.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74280 Milliseconds Vbucket 117 2016-04-07T03:42:00.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74310 Milliseconds Vbucket 117 2016-04-07T03:42:00.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74340 Milliseconds Vbucket 117 2016-04-07T03:42:00.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74370 Milliseconds Vbucket 117 2016-04-07T03:42:00.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74400 Milliseconds Vbucket 117 2016-04-07T03:42:00.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74430 Milliseconds Vbucket 117 2016-04-07T03:42:00.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74460 Milliseconds Vbucket 117 2016-04-07T03:42:00.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74490 Milliseconds Vbucket 117 2016-04-07T03:42:00.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74520 Milliseconds Vbucket 117 2016-04-07T03:42:00.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74550 Milliseconds Vbucket 117 2016-04-07T03:42:00.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74580 Milliseconds Vbucket 117 2016-04-07T03:42:00.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74610 Milliseconds Vbucket 117 2016-04-07T03:42:00.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74640 Milliseconds Vbucket 117 2016-04-07T03:42:00.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74670 Milliseconds Vbucket 117 2016-04-07T03:42:00.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74700 Milliseconds Vbucket 117 2016-04-07T03:42:00.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74730 Milliseconds Vbucket 117 2016-04-07T03:42:00.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74760 Milliseconds Vbucket 117 2016-04-07T03:42:00.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74790 Milliseconds Vbucket 117 2016-04-07T03:42:00.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74820 Milliseconds Vbucket 117 2016-04-07T03:42:00.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74850 Milliseconds Vbucket 117 2016-04-07T03:42:00.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74880 Milliseconds Vbucket 117 2016-04-07T03:42:00.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74910 Milliseconds Vbucket 117 2016-04-07T03:42:00.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74940 Milliseconds Vbucket 117 2016-04-07T03:42:00.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74970 Milliseconds Vbucket 117 2016-04-07T03:42:00.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75000 Milliseconds Vbucket 117 2016-04-07T03:42:00.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75030 Milliseconds Vbucket 117 2016-04-07T03:42:00.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75060 Milliseconds Vbucket 117 2016-04-07T03:42:00.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75090 Milliseconds Vbucket 117 2016-04-07T03:42:00.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75120 Milliseconds Vbucket 117 2016-04-07T03:42:00.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75150 Milliseconds Vbucket 117 2016-04-07T03:42:00.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75180 Milliseconds Vbucket 117 2016-04-07T03:42:00.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75210 Milliseconds Vbucket 117 2016-04-07T03:42:00.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75240 Milliseconds Vbucket 117 2016-04-07T03:42:00.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75270 Milliseconds Vbucket 117 2016-04-07T03:42:00.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75300 Milliseconds Vbucket 117 2016-04-07T03:42:01.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75330 Milliseconds Vbucket 117 2016-04-07T03:42:01.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75360 Milliseconds Vbucket 117 2016-04-07T03:42:01.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75390 Milliseconds Vbucket 117 2016-04-07T03:42:01.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75420 Milliseconds Vbucket 117 2016-04-07T03:42:01.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75450 Milliseconds Vbucket 117 2016-04-07T03:42:01.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75480 Milliseconds Vbucket 117 2016-04-07T03:42:01.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75510 Milliseconds Vbucket 117 2016-04-07T03:42:01.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75540 Milliseconds Vbucket 117 2016-04-07T03:42:01.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75570 Milliseconds Vbucket 117 2016-04-07T03:42:01.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75600 Milliseconds Vbucket 117 2016-04-07T03:42:01.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75630 Milliseconds Vbucket 117 2016-04-07T03:42:01.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75660 Milliseconds Vbucket 117 2016-04-07T03:42:01.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75690 Milliseconds Vbucket 117 2016-04-07T03:42:01.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75720 Milliseconds Vbucket 117 2016-04-07T03:42:01.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75750 Milliseconds Vbucket 117 2016-04-07T03:42:01.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75780 Milliseconds Vbucket 117 2016-04-07T03:42:01.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75810 Milliseconds Vbucket 117 2016-04-07T03:42:01.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75840 Milliseconds Vbucket 117 2016-04-07T03:42:01.551-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4200000 QueuedCount 49999 2016-04-07T03:42:01.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75870 Milliseconds Vbucket 117 2016-04-07T03:42:01.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75900 Milliseconds Vbucket 117 2016-04-07T03:42:01.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75930 Milliseconds Vbucket 117 2016-04-07T03:42:01.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75960 Milliseconds Vbucket 117 2016-04-07T03:42:01.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75990 Milliseconds Vbucket 117 2016-04-07T03:42:01.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76020 Milliseconds Vbucket 117 2016-04-07T03:42:01.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76050 Milliseconds Vbucket 117 2016-04-07T03:42:01.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76080 Milliseconds Vbucket 117 2016-04-07T03:42:01.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76110 Milliseconds Vbucket 117 2016-04-07T03:42:01.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76140 Milliseconds Vbucket 117 2016-04-07T03:42:01.852-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4210000 QueuedCount 50000 2016-04-07T03:42:01.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76170 Milliseconds Vbucket 117 2016-04-07T03:42:01.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76200 Milliseconds Vbucket 117 2016-04-07T03:42:01.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76230 Milliseconds Vbucket 117 2016-04-07T03:42:01.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76260 Milliseconds Vbucket 117 2016-04-07T03:42:01.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76290 Milliseconds Vbucket 117 2016-04-07T03:42:02.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76320 Milliseconds Vbucket 117 2016-04-07T03:42:02.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76350 Milliseconds Vbucket 117 2016-04-07T03:42:02.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76380 Milliseconds Vbucket 117 2016-04-07T03:42:02.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76410 Milliseconds Vbucket 117 2016-04-07T03:42:02.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76440 Milliseconds Vbucket 117 2016-04-07T03:42:02.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76470 Milliseconds Vbucket 117 2016-04-07T03:42:02.192-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4220000 QueuedCount 45391 2016-04-07T03:42:02.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76500 Milliseconds Vbucket 117 2016-04-07T03:42:02.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76530 Milliseconds Vbucket 117 2016-04-07T03:42:02.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76560 Milliseconds Vbucket 117 2016-04-07T03:42:02.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76590 Milliseconds Vbucket 117 2016-04-07T03:42:02.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76620 Milliseconds Vbucket 117 2016-04-07T03:42:02.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76650 Milliseconds Vbucket 117 2016-04-07T03:42:02.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76680 Milliseconds Vbucket 117 2016-04-07T03:42:02.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76710 Milliseconds Vbucket 117 2016-04-07T03:42:02.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76740 Milliseconds Vbucket 117 2016-04-07T03:42:02.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76770 Milliseconds Vbucket 117 2016-04-07T03:42:02.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76800 Milliseconds Vbucket 117 2016-04-07T03:42:02.508-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4230000 QueuedCount 35391 2016-04-07T03:42:02.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76830 Milliseconds Vbucket 117 2016-04-07T03:42:02.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76860 Milliseconds Vbucket 117 2016-04-07T03:42:02.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76890 Milliseconds Vbucket 117 2016-04-07T03:42:02.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76920 Milliseconds Vbucket 117 2016-04-07T03:42:02.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76950 Milliseconds Vbucket 117 2016-04-07T03:42:02.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76980 Milliseconds Vbucket 117 2016-04-07T03:42:02.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77010 Milliseconds Vbucket 117 2016-04-07T03:42:02.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77040 Milliseconds Vbucket 117 2016-04-07T03:42:02.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77070 Milliseconds Vbucket 117 2016-04-07T03:42:02.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77100 Milliseconds Vbucket 117 2016-04-07T03:42:02.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77130 Milliseconds Vbucket 117 2016-04-07T03:42:02.838-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4240000 QueuedCount 25391 2016-04-07T03:42:02.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77160 Milliseconds Vbucket 117 2016-04-07T03:42:02.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77190 Milliseconds Vbucket 117 2016-04-07T03:42:02.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77220 Milliseconds Vbucket 117 2016-04-07T03:42:02.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77250 Milliseconds Vbucket 117 2016-04-07T03:42:02.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77280 Milliseconds Vbucket 117 2016-04-07T03:42:03.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77310 Milliseconds Vbucket 117 2016-04-07T03:42:03.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77340 Milliseconds Vbucket 117 2016-04-07T03:42:03.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77370 Milliseconds Vbucket 117 2016-04-07T03:42:03.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77400 Milliseconds Vbucket 117 2016-04-07T03:42:03.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77430 Milliseconds Vbucket 117 2016-04-07T03:42:03.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77460 Milliseconds Vbucket 117 2016-04-07T03:42:03.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77490 Milliseconds Vbucket 117 2016-04-07T03:42:03.209-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4250000 QueuedCount 15391 2016-04-07T03:42:03.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77520 Milliseconds Vbucket 117 2016-04-07T03:42:03.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77550 Milliseconds Vbucket 117 2016-04-07T03:42:03.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77580 Milliseconds Vbucket 117 2016-04-07T03:42:03.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77610 Milliseconds Vbucket 117 2016-04-07T03:42:03.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77640 Milliseconds Vbucket 117 2016-04-07T03:42:03.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77670 Milliseconds Vbucket 117 2016-04-07T03:42:03.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77700 Milliseconds Vbucket 117 2016-04-07T03:42:03.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77730 Milliseconds Vbucket 117 2016-04-07T03:42:03.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77760 Milliseconds Vbucket 117 2016-04-07T03:42:03.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77790 Milliseconds Vbucket 117 2016-04-07T03:42:03.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77820 Milliseconds Vbucket 117 2016-04-07T03:42:03.54-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4260000 QueuedCount 5391 2016-04-07T03:42:03.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77850 Milliseconds Vbucket 117 2016-04-07T03:42:03.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77880 Milliseconds Vbucket 117 2016-04-07T03:42:03.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77910 Milliseconds Vbucket 117 2016-04-07T03:42:03.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77940 Milliseconds Vbucket 117 2016-04-07T03:42:03.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77970 Milliseconds Vbucket 117 2016-04-07T03:42:03.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78000 Milliseconds Vbucket 117 2016-04-07T03:42:03.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78030 Milliseconds Vbucket 117 2016-04-07T03:42:03.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78060 Milliseconds Vbucket 117 2016-04-07T03:42:03.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78090 Milliseconds Vbucket 117 2016-04-07T03:42:03.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78120 Milliseconds Vbucket 117 2016-04-07T03:42:03.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78150 Milliseconds Vbucket 117 2016-04-07T03:42:03.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78180 Milliseconds Vbucket 117 2016-04-07T03:42:03.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78210 Milliseconds Vbucket 117 2016-04-07T03:42:03.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78240 Milliseconds Vbucket 117 2016-04-07T03:42:03.95-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.261µs CommitTime 258.188062ms TotalFlushTime 2m21.116181761s TotalCommitTime 5.278831933s 2016-04-07T03:42:03.95-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 4265391, 4265391, 25 committed:true 2016-04-07T03:42:03.95-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 4265391, 4265391, 25 committed:true) SnapCreateDur 269.526024ms SnapOpenDur 40.533µs 2016-04-07T03:42:03.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78270 Milliseconds Vbucket 117 2016-04-07T03:42:03.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78300 Milliseconds Vbucket 117 2016-04-07T03:42:04.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78330 Milliseconds Vbucket 117 2016-04-07T03:42:04.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78360 Milliseconds Vbucket 117 2016-04-07T03:42:04.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78390 Milliseconds Vbucket 117 2016-04-07T03:42:04.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78420 Milliseconds Vbucket 117 2016-04-07T03:42:04.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78450 Milliseconds Vbucket 117 2016-04-07T03:42:04.148-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4270000 QueuedCount 29364 2016-04-07T03:42:04.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78480 Milliseconds Vbucket 117 2016-04-07T03:42:04.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78510 Milliseconds Vbucket 117 2016-04-07T03:42:04.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78540 Milliseconds Vbucket 117 2016-04-07T03:42:04.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78570 Milliseconds Vbucket 117 2016-04-07T03:42:04.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78600 Milliseconds Vbucket 117 2016-04-07T03:42:04.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78630 Milliseconds Vbucket 117 2016-04-07T03:42:04.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78660 Milliseconds Vbucket 117 2016-04-07T03:42:04.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78690 Milliseconds Vbucket 117 2016-04-07T03:42:04.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78720 Milliseconds Vbucket 117 2016-04-07T03:42:04.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78750 Milliseconds Vbucket 117 2016-04-07T03:42:04.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78780 Milliseconds Vbucket 117 2016-04-07T03:42:04.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78810 Milliseconds Vbucket 117 2016-04-07T03:42:04.523-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4280000 QueuedCount 19364 2016-04-07T03:42:04.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78840 Milliseconds Vbucket 117 2016-04-07T03:42:04.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78870 Milliseconds Vbucket 117 2016-04-07T03:42:04.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78900 Milliseconds Vbucket 117 2016-04-07T03:42:04.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78930 Milliseconds Vbucket 117 2016-04-07T03:42:04.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78960 Milliseconds Vbucket 117 2016-04-07T03:42:04.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78990 Milliseconds Vbucket 117 2016-04-07T03:42:04.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79020 Milliseconds Vbucket 117 2016-04-07T03:42:04.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79050 Milliseconds Vbucket 117 2016-04-07T03:42:04.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79080 Milliseconds Vbucket 117 2016-04-07T03:42:04.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79110 Milliseconds Vbucket 117 2016-04-07T03:42:04.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79140 Milliseconds Vbucket 117 2016-04-07T03:42:04.862-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4290000 QueuedCount 9364 2016-04-07T03:42:04.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79170 Milliseconds Vbucket 117 2016-04-07T03:42:04.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79200 Milliseconds Vbucket 117 2016-04-07T03:42:04.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79230 Milliseconds Vbucket 117 2016-04-07T03:42:04.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79260 Milliseconds Vbucket 117 2016-04-07T03:42:04.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79290 Milliseconds Vbucket 117 2016-04-07T03:42:05.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79320 Milliseconds Vbucket 117 2016-04-07T03:42:05.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79350 Milliseconds Vbucket 117 2016-04-07T03:42:05.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79380 Milliseconds Vbucket 117 2016-04-07T03:42:05.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79410 Milliseconds Vbucket 117 2016-04-07T03:42:05.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79440 Milliseconds Vbucket 117 2016-04-07T03:42:05.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79470 Milliseconds Vbucket 117 2016-04-07T03:42:05.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79500 Milliseconds Vbucket 117 2016-04-07T03:42:05.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79530 Milliseconds Vbucket 117 2016-04-07T03:42:05.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79560 Milliseconds Vbucket 117 2016-04-07T03:42:05.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79590 Milliseconds Vbucket 117 2016-04-07T03:42:05.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79620 Milliseconds Vbucket 117 2016-04-07T03:42:05.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79650 Milliseconds Vbucket 117 2016-04-07T03:42:05.373-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.811µs CommitTime 195.053356ms TotalFlushTime 2m22.286128066s TotalCommitTime 5.473885289s 2016-04-07T03:42:05.373-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 4299364, 4299364, 26 committed:true 2016-04-07T03:42:05.373-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 4299364, 4299364, 26 committed:true) SnapCreateDur 205.98437ms SnapOpenDur 36.528µs 2016-04-07T03:42:05.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79680 Milliseconds Vbucket 117 2016-04-07T03:42:05.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79710 Milliseconds Vbucket 117 2016-04-07T03:42:05.407-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4300000 QueuedCount 21100 2016-04-07T03:42:05.416-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:05.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79740 Milliseconds Vbucket 117 2016-04-07T03:42:05.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79770 Milliseconds Vbucket 117 2016-04-07T03:42:05.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79800 Milliseconds Vbucket 117 2016-04-07T03:42:05.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79830 Milliseconds Vbucket 117 2016-04-07T03:42:05.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79860 Milliseconds Vbucket 117 2016-04-07T03:42:05.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79890 Milliseconds Vbucket 117 2016-04-07T03:42:05.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79920 Milliseconds Vbucket 117 2016-04-07T03:42:05.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79950 Milliseconds Vbucket 117 2016-04-07T03:42:05.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79980 Milliseconds Vbucket 117 2016-04-07T03:42:05.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80010 Milliseconds Vbucket 117 2016-04-07T03:42:05.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80040 Milliseconds Vbucket 117 2016-04-07T03:42:05.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80070 Milliseconds Vbucket 117 2016-04-07T03:42:05.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80100 Milliseconds Vbucket 117 2016-04-07T03:42:05.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80130 Milliseconds Vbucket 117 2016-04-07T03:42:05.852-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4310000 QueuedCount 50000 2016-04-07T03:42:05.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80160 Milliseconds Vbucket 117 2016-04-07T03:42:05.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80190 Milliseconds Vbucket 117 2016-04-07T03:42:05.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80220 Milliseconds Vbucket 117 2016-04-07T03:42:05.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80250 Milliseconds Vbucket 117 2016-04-07T03:42:05.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80280 Milliseconds Vbucket 117 2016-04-07T03:42:06.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80310 Milliseconds Vbucket 117 2016-04-07T03:42:06.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80340 Milliseconds Vbucket 117 2016-04-07T03:42:06.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80370 Milliseconds Vbucket 117 2016-04-07T03:42:06.094-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:06.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80400 Milliseconds Vbucket 117 2016-04-07T03:42:06.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80430 Milliseconds Vbucket 117 2016-04-07T03:42:06.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80460 Milliseconds Vbucket 117 2016-04-07T03:42:06.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80490 Milliseconds Vbucket 117 2016-04-07T03:42:06.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80520 Milliseconds Vbucket 117 2016-04-07T03:42:06.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80550 Milliseconds Vbucket 117 2016-04-07T03:42:06.244-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4320000 QueuedCount 50000 2016-04-07T03:42:06.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80580 Milliseconds Vbucket 117 2016-04-07T03:42:06.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80610 Milliseconds Vbucket 117 2016-04-07T03:42:06.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80640 Milliseconds Vbucket 117 2016-04-07T03:42:06.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80670 Milliseconds Vbucket 117 2016-04-07T03:42:06.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80700 Milliseconds Vbucket 117 2016-04-07T03:42:06.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80730 Milliseconds Vbucket 117 2016-04-07T03:42:06.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80760 Milliseconds Vbucket 117 2016-04-07T03:42:06.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80790 Milliseconds Vbucket 117 2016-04-07T03:42:06.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80820 Milliseconds Vbucket 117 2016-04-07T03:42:06.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80850 Milliseconds Vbucket 117 2016-04-07T03:42:06.56-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4330000 QueuedCount 50000 2016-04-07T03:42:06.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80880 Milliseconds Vbucket 117 2016-04-07T03:42:06.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80910 Milliseconds Vbucket 117 2016-04-07T03:42:06.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80940 Milliseconds Vbucket 117 2016-04-07T03:42:06.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80970 Milliseconds Vbucket 117 2016-04-07T03:42:06.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81000 Milliseconds Vbucket 117 2016-04-07T03:42:06.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81030 Milliseconds Vbucket 117 2016-04-07T03:42:06.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81060 Milliseconds Vbucket 117 2016-04-07T03:42:06.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81090 Milliseconds Vbucket 117 2016-04-07T03:42:06.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81120 Milliseconds Vbucket 117 2016-04-07T03:42:06.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81150 Milliseconds Vbucket 117 2016-04-07T03:42:06.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81180 Milliseconds Vbucket 117 2016-04-07T03:42:06.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81210 Milliseconds Vbucket 117 2016-04-07T03:42:06.912-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4340000 QueuedCount 50000 2016-04-07T03:42:06.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81240 Milliseconds Vbucket 117 2016-04-07T03:42:06.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81270 Milliseconds Vbucket 117 2016-04-07T03:42:06.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81300 Milliseconds Vbucket 117 2016-04-07T03:42:07.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81330 Milliseconds Vbucket 117 2016-04-07T03:42:07.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81360 Milliseconds Vbucket 117 2016-04-07T03:42:07.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81390 Milliseconds Vbucket 117 2016-04-07T03:42:07.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81420 Milliseconds Vbucket 117 2016-04-07T03:42:07.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81450 Milliseconds Vbucket 117 2016-04-07T03:42:07.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81480 Milliseconds Vbucket 117 2016-04-07T03:42:07.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81510 Milliseconds Vbucket 117 2016-04-07T03:42:07.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81540 Milliseconds Vbucket 117 2016-04-07T03:42:07.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81570 Milliseconds Vbucket 117 2016-04-07T03:42:07.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81600 Milliseconds Vbucket 117 2016-04-07T03:42:07.302-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4350000 QueuedCount 50000 2016-04-07T03:42:07.305-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:07.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81630 Milliseconds Vbucket 117 2016-04-07T03:42:07.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81660 Milliseconds Vbucket 117 2016-04-07T03:42:07.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81690 Milliseconds Vbucket 117 2016-04-07T03:42:07.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81720 Milliseconds Vbucket 117 2016-04-07T03:42:07.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81750 Milliseconds Vbucket 117 2016-04-07T03:42:07.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81780 Milliseconds Vbucket 117 2016-04-07T03:42:07.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81810 Milliseconds Vbucket 117 2016-04-07T03:42:07.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81840 Milliseconds Vbucket 117 2016-04-07T03:42:07.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81870 Milliseconds Vbucket 117 2016-04-07T03:42:07.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81900 Milliseconds Vbucket 117 2016-04-07T03:42:07.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81930 Milliseconds Vbucket 117 2016-04-07T03:42:07.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81960 Milliseconds Vbucket 117 2016-04-07T03:42:07.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81990 Milliseconds Vbucket 117 2016-04-07T03:42:07.698-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4360000 QueuedCount 50000 2016-04-07T03:42:07.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82020 Milliseconds Vbucket 117 2016-04-07T03:42:07.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82050 Milliseconds Vbucket 117 2016-04-07T03:42:07.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82080 Milliseconds Vbucket 117 2016-04-07T03:42:07.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82110 Milliseconds Vbucket 117 2016-04-07T03:42:07.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82140 Milliseconds Vbucket 117 2016-04-07T03:42:07.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82170 Milliseconds Vbucket 117 2016-04-07T03:42:07.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82200 Milliseconds Vbucket 117 2016-04-07T03:42:07.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82230 Milliseconds Vbucket 117 2016-04-07T03:42:07.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82260 Milliseconds Vbucket 117 2016-04-07T03:42:07.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82290 Milliseconds Vbucket 117 2016-04-07T03:42:08.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82320 Milliseconds Vbucket 117 2016-04-07T03:42:08.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82350 Milliseconds Vbucket 117 2016-04-07T03:42:08.064-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4370000 QueuedCount 50000 2016-04-07T03:42:08.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82380 Milliseconds Vbucket 117 2016-04-07T03:42:08.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82410 Milliseconds Vbucket 117 2016-04-07T03:42:08.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82440 Milliseconds Vbucket 117 2016-04-07T03:42:08.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82470 Milliseconds Vbucket 117 2016-04-07T03:42:08.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82500 Milliseconds Vbucket 117 2016-04-07T03:42:08.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82530 Milliseconds Vbucket 117 2016-04-07T03:42:08.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82560 Milliseconds Vbucket 117 2016-04-07T03:42:08.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82590 Milliseconds Vbucket 117 2016-04-07T03:42:08.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82620 Milliseconds Vbucket 117 2016-04-07T03:42:08.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82650 Milliseconds Vbucket 117 2016-04-07T03:42:08.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82680 Milliseconds Vbucket 117 2016-04-07T03:42:08.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82710 Milliseconds Vbucket 117 2016-04-07T03:42:08.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82740 Milliseconds Vbucket 117 2016-04-07T03:42:08.443-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4380000 QueuedCount 50000 2016-04-07T03:42:08.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82770 Milliseconds Vbucket 117 2016-04-07T03:42:08.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82800 Milliseconds Vbucket 117 2016-04-07T03:42:08.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82830 Milliseconds Vbucket 117 2016-04-07T03:42:08.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82860 Milliseconds Vbucket 117 2016-04-07T03:42:08.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82890 Milliseconds Vbucket 117 2016-04-07T03:42:08.586-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:08.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82920 Milliseconds Vbucket 117 2016-04-07T03:42:08.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82950 Milliseconds Vbucket 117 2016-04-07T03:42:08.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82980 Milliseconds Vbucket 117 2016-04-07T03:42:08.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83010 Milliseconds Vbucket 117 2016-04-07T03:42:08.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83040 Milliseconds Vbucket 117 2016-04-07T03:42:08.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83070 Milliseconds Vbucket 117 2016-04-07T03:42:08.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83100 Milliseconds Vbucket 117 2016-04-07T03:42:08.814-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4390000 QueuedCount 50000 2016-04-07T03:42:08.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83130 Milliseconds Vbucket 117 2016-04-07T03:42:08.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83160 Milliseconds Vbucket 117 2016-04-07T03:42:08.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83190 Milliseconds Vbucket 117 2016-04-07T03:42:08.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83220 Milliseconds Vbucket 117 2016-04-07T03:42:08.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83250 Milliseconds Vbucket 117 2016-04-07T03:42:08.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83280 Milliseconds Vbucket 117 2016-04-07T03:42:09.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83310 Milliseconds Vbucket 117 2016-04-07T03:42:09.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83340 Milliseconds Vbucket 117 2016-04-07T03:42:09.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83370 Milliseconds Vbucket 117 2016-04-07T03:42:09.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83400 Milliseconds Vbucket 117 2016-04-07T03:42:09.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83430 Milliseconds Vbucket 117 2016-04-07T03:42:09.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83460 Milliseconds Vbucket 117 2016-04-07T03:42:09.184-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4400000 QueuedCount 49993 2016-04-07T03:42:09.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83490 Milliseconds Vbucket 117 2016-04-07T03:42:09.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83520 Milliseconds Vbucket 117 2016-04-07T03:42:09.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83550 Milliseconds Vbucket 117 2016-04-07T03:42:09.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83580 Milliseconds Vbucket 117 2016-04-07T03:42:09.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83610 Milliseconds Vbucket 117 2016-04-07T03:42:09.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83640 Milliseconds Vbucket 117 2016-04-07T03:42:09.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83670 Milliseconds Vbucket 117 2016-04-07T03:42:09.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83700 Milliseconds Vbucket 117 2016-04-07T03:42:09.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83730 Milliseconds Vbucket 117 2016-04-07T03:42:09.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83760 Milliseconds Vbucket 117 2016-04-07T03:42:09.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83790 Milliseconds Vbucket 117 2016-04-07T03:42:09.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83820 Milliseconds Vbucket 117 2016-04-07T03:42:09.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83850 Milliseconds Vbucket 117 2016-04-07T03:42:09.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83880 Milliseconds Vbucket 117 2016-04-07T03:42:09.577-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4410000 QueuedCount 49999 2016-04-07T03:42:09.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83910 Milliseconds Vbucket 117 2016-04-07T03:42:09.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83940 Milliseconds Vbucket 117 2016-04-07T03:42:09.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83970 Milliseconds Vbucket 117 2016-04-07T03:42:09.687-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:09.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84000 Milliseconds Vbucket 117 2016-04-07T03:42:09.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84030 Milliseconds Vbucket 117 2016-04-07T03:42:09.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84060 Milliseconds Vbucket 117 2016-04-07T03:42:09.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84090 Milliseconds Vbucket 117 2016-04-07T03:42:09.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84120 Milliseconds Vbucket 117 2016-04-07T03:42:09.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84150 Milliseconds Vbucket 117 2016-04-07T03:42:09.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84180 Milliseconds Vbucket 117 2016-04-07T03:42:09.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84210 Milliseconds Vbucket 117 2016-04-07T03:42:09.92-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4420000 QueuedCount 50000 2016-04-07T03:42:09.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84240 Milliseconds Vbucket 117 2016-04-07T03:42:09.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84270 Milliseconds Vbucket 117 2016-04-07T03:42:09.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84300 Milliseconds Vbucket 117 2016-04-07T03:42:10.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84330 Milliseconds Vbucket 117 2016-04-07T03:42:10.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84360 Milliseconds Vbucket 117 2016-04-07T03:42:10.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84390 Milliseconds Vbucket 117 2016-04-07T03:42:10.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84420 Milliseconds Vbucket 117 2016-04-07T03:42:10.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84450 Milliseconds Vbucket 117 2016-04-07T03:42:10.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84480 Milliseconds Vbucket 117 2016-04-07T03:42:10.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84510 Milliseconds Vbucket 117 2016-04-07T03:42:10.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84540 Milliseconds Vbucket 117 2016-04-07T03:42:10.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84570 Milliseconds Vbucket 117 2016-04-07T03:42:10.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84600 Milliseconds Vbucket 117 2016-04-07T03:42:10.301-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4430000 QueuedCount 50000 2016-04-07T03:42:10.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84630 Milliseconds Vbucket 117 2016-04-07T03:42:10.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84660 Milliseconds Vbucket 117 2016-04-07T03:42:10.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84690 Milliseconds Vbucket 117 2016-04-07T03:42:10.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84720 Milliseconds Vbucket 117 2016-04-07T03:42:10.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84750 Milliseconds Vbucket 117 2016-04-07T03:42:10.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84780 Milliseconds Vbucket 117 2016-04-07T03:42:10.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84810 Milliseconds Vbucket 117 2016-04-07T03:42:10.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84840 Milliseconds Vbucket 117 2016-04-07T03:42:10.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84870 Milliseconds Vbucket 117 2016-04-07T03:42:10.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84900 Milliseconds Vbucket 117 2016-04-07T03:42:10.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84930 Milliseconds Vbucket 117 2016-04-07T03:42:10.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84960 Milliseconds Vbucket 117 2016-04-07T03:42:10.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84990 Milliseconds Vbucket 117 2016-04-07T03:42:10.703-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4440000 QueuedCount 50000 2016-04-07T03:42:10.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85020 Milliseconds Vbucket 117 2016-04-07T03:42:10.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85050 Milliseconds Vbucket 117 2016-04-07T03:42:10.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85080 Milliseconds Vbucket 117 2016-04-07T03:42:10.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85110 Milliseconds Vbucket 117 2016-04-07T03:42:10.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85140 Milliseconds Vbucket 117 2016-04-07T03:42:10.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85170 Milliseconds Vbucket 117 2016-04-07T03:42:10.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85200 Milliseconds Vbucket 117 2016-04-07T03:42:10.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85230 Milliseconds Vbucket 117 2016-04-07T03:42:10.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85260 Milliseconds Vbucket 117 2016-04-07T03:42:10.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85290 Milliseconds Vbucket 117 2016-04-07T03:42:11.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85320 Milliseconds Vbucket 117 2016-04-07T03:42:11.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85350 Milliseconds Vbucket 117 2016-04-07T03:42:11.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85380 Milliseconds Vbucket 117 2016-04-07T03:42:11.102-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4450000 QueuedCount 42969 2016-04-07T03:42:11.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85410 Milliseconds Vbucket 117 2016-04-07T03:42:11.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85440 Milliseconds Vbucket 117 2016-04-07T03:42:11.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85470 Milliseconds Vbucket 117 2016-04-07T03:42:11.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85500 Milliseconds Vbucket 117 2016-04-07T03:42:11.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85530 Milliseconds Vbucket 117 2016-04-07T03:42:11.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85560 Milliseconds Vbucket 117 2016-04-07T03:42:11.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85590 Milliseconds Vbucket 117 2016-04-07T03:42:11.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85620 Milliseconds Vbucket 117 2016-04-07T03:42:11.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85650 Milliseconds Vbucket 117 2016-04-07T03:42:11.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85680 Milliseconds Vbucket 117 2016-04-07T03:42:11.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85710 Milliseconds Vbucket 117 2016-04-07T03:42:11.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85740 Milliseconds Vbucket 117 2016-04-07T03:42:11.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85770 Milliseconds Vbucket 117 2016-04-07T03:42:11.48-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4460000 QueuedCount 32969 2016-04-07T03:42:11.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85800 Milliseconds Vbucket 117 2016-04-07T03:42:11.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85830 Milliseconds Vbucket 117 2016-04-07T03:42:11.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85860 Milliseconds Vbucket 117 2016-04-07T03:42:11.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85890 Milliseconds Vbucket 117 2016-04-07T03:42:11.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85920 Milliseconds Vbucket 117 2016-04-07T03:42:11.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85950 Milliseconds Vbucket 117 2016-04-07T03:42:11.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85980 Milliseconds Vbucket 117 2016-04-07T03:42:11.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86010 Milliseconds Vbucket 117 2016-04-07T03:42:11.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86040 Milliseconds Vbucket 117 2016-04-07T03:42:11.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86070 Milliseconds Vbucket 117 2016-04-07T03:42:11.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86100 Milliseconds Vbucket 117 2016-04-07T03:42:11.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86130 Milliseconds Vbucket 117 2016-04-07T03:42:11.853-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4470000 QueuedCount 22969 2016-04-07T03:42:11.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86160 Milliseconds Vbucket 117 2016-04-07T03:42:11.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86190 Milliseconds Vbucket 117 2016-04-07T03:42:11.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86220 Milliseconds Vbucket 117 2016-04-07T03:42:11.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86250 Milliseconds Vbucket 117 2016-04-07T03:42:11.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86280 Milliseconds Vbucket 117 2016-04-07T03:42:12.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86310 Milliseconds Vbucket 117 2016-04-07T03:42:12.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86340 Milliseconds Vbucket 117 2016-04-07T03:42:12.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86370 Milliseconds Vbucket 117 2016-04-07T03:42:12.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86400 Milliseconds Vbucket 117 2016-04-07T03:42:12.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86430 Milliseconds Vbucket 117 2016-04-07T03:42:12.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86460 Milliseconds Vbucket 117 2016-04-07T03:42:12.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86490 Milliseconds Vbucket 117 2016-04-07T03:42:12.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86520 Milliseconds Vbucket 117 2016-04-07T03:42:12.214-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4480000 QueuedCount 12969 2016-04-07T03:42:12.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86550 Milliseconds Vbucket 117 2016-04-07T03:42:12.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86580 Milliseconds Vbucket 117 2016-04-07T03:42:12.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86610 Milliseconds Vbucket 117 2016-04-07T03:42:12.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86640 Milliseconds Vbucket 117 2016-04-07T03:42:12.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86670 Milliseconds Vbucket 117 2016-04-07T03:42:12.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86700 Milliseconds Vbucket 117 2016-04-07T03:42:12.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86730 Milliseconds Vbucket 117 2016-04-07T03:42:12.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86760 Milliseconds Vbucket 117 2016-04-07T03:42:12.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86790 Milliseconds Vbucket 117 2016-04-07T03:42:12.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86820 Milliseconds Vbucket 117 2016-04-07T03:42:12.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86850 Milliseconds Vbucket 117 2016-04-07T03:42:12.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86880 Milliseconds Vbucket 117 2016-04-07T03:42:12.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86910 Milliseconds Vbucket 117 2016-04-07T03:42:12.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86940 Milliseconds Vbucket 117 2016-04-07T03:42:12.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86970 Milliseconds Vbucket 117 2016-04-07T03:42:12.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87000 Milliseconds Vbucket 117 2016-04-07T03:42:12.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87030 Milliseconds Vbucket 117 2016-04-07T03:42:12.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87060 Milliseconds Vbucket 117 2016-04-07T03:42:12.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87090 Milliseconds Vbucket 117 2016-04-07T03:42:12.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87120 Milliseconds Vbucket 117 2016-04-07T03:42:12.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87150 Milliseconds Vbucket 117 2016-04-07T03:42:12.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87180 Milliseconds Vbucket 117 2016-04-07T03:42:12.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87210 Milliseconds Vbucket 117 2016-04-07T03:42:12.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87240 Milliseconds Vbucket 117 2016-04-07T03:42:12.94-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":6834820,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":4114337088,"bucket-1:myindex:avg_ts_items_count":83507,"bucket-1:myindex:build_progress":22,"bucket-1:myindex:data_size":1424347655,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":3955392512,"bucket-1:myindex:flush_queue_size":4153,"bucket-1:myindex:frag_percent":63,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":470146215,"bucket-1:myindex:items_count":4299364,"bucket-1:myindex:num_commits":26,"bucket-1:myindex:num_compactions":1,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":4463982,"bucket-1:myindex:num_docs_pending":8672209,"bucket-1:myindex:num_docs_queued":6863809,"bucket-1:myindex:num_flush_queued":4492969,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":26,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1423197112,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"26 5473885289 1280760960565697187","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"4488816 23791335571 28722739202172197","bucket-1:myindex:timings/storage_handle_open":"26 636814 16154303768","bucket-1:myindex:timings/storage_info":"105 859167 16508218625","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"26 24163002 266453858988048","bucket-1:myindex:timings/storage_meta_set":"26 9608136 3741513131842","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"8977632 122484177637 3953479139845165053","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":11327789,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":223,"memory_quota":31457280000,"memory_used":5582180352,"needs_restart":false,"num_connections":0,"timings/stats_response":"1345 104958953 14952635905451"} 2016-04-07T03:42:12.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87270 Milliseconds Vbucket 117 2016-04-07T03:42:12.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87300 Milliseconds Vbucket 117 2016-04-07T03:42:13.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87330 Milliseconds Vbucket 117 2016-04-07T03:42:13.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87360 Milliseconds Vbucket 117 2016-04-07T03:42:13.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87390 Milliseconds Vbucket 117 2016-04-07T03:42:13.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87420 Milliseconds Vbucket 117 2016-04-07T03:42:13.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87450 Milliseconds Vbucket 117 2016-04-07T03:42:13.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87480 Milliseconds Vbucket 117 2016-04-07T03:42:13.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87510 Milliseconds Vbucket 117 2016-04-07T03:42:13.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87540 Milliseconds Vbucket 117 2016-04-07T03:42:13.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87570 Milliseconds Vbucket 117 2016-04-07T03:42:13.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87600 Milliseconds Vbucket 117 2016-04-07T03:42:13.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87630 Milliseconds Vbucket 117 2016-04-07T03:42:13.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87660 Milliseconds Vbucket 117 2016-04-07T03:42:13.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87690 Milliseconds Vbucket 117 2016-04-07T03:42:13.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87720 Milliseconds Vbucket 117 2016-04-07T03:42:13.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87750 Milliseconds Vbucket 117 2016-04-07T03:42:13.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87780 Milliseconds Vbucket 117 2016-04-07T03:42:13.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87810 Milliseconds Vbucket 117 2016-04-07T03:42:13.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87840 Milliseconds Vbucket 117 2016-04-07T03:42:13.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87870 Milliseconds Vbucket 117 2016-04-07T03:42:13.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87900 Milliseconds Vbucket 117 2016-04-07T03:42:13.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87930 Milliseconds Vbucket 117 2016-04-07T03:42:13.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87960 Milliseconds Vbucket 117 2016-04-07T03:42:13.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87990 Milliseconds Vbucket 117 2016-04-07T03:42:13.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88020 Milliseconds Vbucket 117 2016-04-07T03:42:13.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88050 Milliseconds Vbucket 117 2016-04-07T03:42:13.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88080 Milliseconds Vbucket 117 2016-04-07T03:42:13.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88110 Milliseconds Vbucket 117 2016-04-07T03:42:13.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88140 Milliseconds Vbucket 117 2016-04-07T03:42:13.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88170 Milliseconds Vbucket 117 2016-04-07T03:42:13.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88200 Milliseconds Vbucket 117 2016-04-07T03:42:13.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88230 Milliseconds Vbucket 117 2016-04-07T03:42:13.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88260 Milliseconds Vbucket 117 2016-04-07T03:42:13.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88290 Milliseconds Vbucket 117 2016-04-07T03:42:14.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88320 Milliseconds Vbucket 117 2016-04-07T03:42:14.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88350 Milliseconds Vbucket 117 2016-04-07T03:42:14.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88380 Milliseconds Vbucket 117 2016-04-07T03:42:14.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88410 Milliseconds Vbucket 117 2016-04-07T03:42:14.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88440 Milliseconds Vbucket 117 2016-04-07T03:42:14.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88470 Milliseconds Vbucket 117 2016-04-07T03:42:14.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88500 Milliseconds Vbucket 117 2016-04-07T03:42:14.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88530 Milliseconds Vbucket 117 2016-04-07T03:42:14.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88560 Milliseconds Vbucket 117 2016-04-07T03:42:14.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88590 Milliseconds Vbucket 117 2016-04-07T03:42:14.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88620 Milliseconds Vbucket 117 2016-04-07T03:42:14.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88650 Milliseconds Vbucket 117 2016-04-07T03:42:14.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88680 Milliseconds Vbucket 117 2016-04-07T03:42:14.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88710 Milliseconds Vbucket 117 2016-04-07T03:42:14.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88740 Milliseconds Vbucket 117 2016-04-07T03:42:14.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88770 Milliseconds Vbucket 117 2016-04-07T03:42:14.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88800 Milliseconds Vbucket 117 2016-04-07T03:42:14.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88830 Milliseconds Vbucket 117 2016-04-07T03:42:14.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88860 Milliseconds Vbucket 117 2016-04-07T03:42:14.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88890 Milliseconds Vbucket 117 2016-04-07T03:42:14.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88920 Milliseconds Vbucket 117 2016-04-07T03:42:14.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88950 Milliseconds Vbucket 117 2016-04-07T03:42:14.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88980 Milliseconds Vbucket 117 2016-04-07T03:42:14.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89010 Milliseconds Vbucket 117 2016-04-07T03:42:14.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89040 Milliseconds Vbucket 117 2016-04-07T03:42:14.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89070 Milliseconds Vbucket 117 2016-04-07T03:42:14.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89100 Milliseconds Vbucket 117 2016-04-07T03:42:14.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89130 Milliseconds Vbucket 117 2016-04-07T03:42:14.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89160 Milliseconds Vbucket 117 2016-04-07T03:42:14.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89190 Milliseconds Vbucket 117 2016-04-07T03:42:14.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89220 Milliseconds Vbucket 117 2016-04-07T03:42:14.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89250 Milliseconds Vbucket 117 2016-04-07T03:42:14.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89280 Milliseconds Vbucket 117 2016-04-07T03:42:15.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89310 Milliseconds Vbucket 117 2016-04-07T03:42:15.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89340 Milliseconds Vbucket 117 2016-04-07T03:42:15.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89370 Milliseconds Vbucket 117 2016-04-07T03:42:15.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89400 Milliseconds Vbucket 117 2016-04-07T03:42:15.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89430 Milliseconds Vbucket 117 2016-04-07T03:42:15.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89460 Milliseconds Vbucket 117 2016-04-07T03:42:15.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89490 Milliseconds Vbucket 117 2016-04-07T03:42:15.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89520 Milliseconds Vbucket 117 2016-04-07T03:42:15.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89550 Milliseconds Vbucket 117 2016-04-07T03:42:15.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89580 Milliseconds Vbucket 117 2016-04-07T03:42:15.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89610 Milliseconds Vbucket 117 2016-04-07T03:42:15.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89640 Milliseconds Vbucket 117 2016-04-07T03:42:15.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89670 Milliseconds Vbucket 117 2016-04-07T03:42:15.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89700 Milliseconds Vbucket 117 2016-04-07T03:42:15.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89730 Milliseconds Vbucket 117 2016-04-07T03:42:15.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89760 Milliseconds Vbucket 117 2016-04-07T03:42:15.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89790 Milliseconds Vbucket 117 2016-04-07T03:42:15.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89820 Milliseconds Vbucket 117 2016-04-07T03:42:15.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89850 Milliseconds Vbucket 117 2016-04-07T03:42:15.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89880 Milliseconds Vbucket 117 2016-04-07T03:42:15.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89910 Milliseconds Vbucket 117 2016-04-07T03:42:15.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89940 Milliseconds Vbucket 117 2016-04-07T03:42:15.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89970 Milliseconds Vbucket 117 2016-04-07T03:42:15.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90000 Milliseconds Vbucket 117 2016-04-07T03:42:15.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90030 Milliseconds Vbucket 117 2016-04-07T03:42:15.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90060 Milliseconds Vbucket 117 2016-04-07T03:42:15.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90090 Milliseconds Vbucket 117 2016-04-07T03:42:15.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90120 Milliseconds Vbucket 117 2016-04-07T03:42:15.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90150 Milliseconds Vbucket 117 2016-04-07T03:42:15.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90180 Milliseconds Vbucket 117 2016-04-07T03:42:15.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90210 Milliseconds Vbucket 117 2016-04-07T03:42:15.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90240 Milliseconds Vbucket 117 2016-04-07T03:42:15.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90270 Milliseconds Vbucket 117 2016-04-07T03:42:15.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90300 Milliseconds Vbucket 117 2016-04-07T03:42:16.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90330 Milliseconds Vbucket 117 2016-04-07T03:42:16.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90360 Milliseconds Vbucket 117 2016-04-07T03:42:16.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90390 Milliseconds Vbucket 117 2016-04-07T03:42:16.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90420 Milliseconds Vbucket 117 2016-04-07T03:42:16.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90450 Milliseconds Vbucket 117 2016-04-07T03:42:16.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90480 Milliseconds Vbucket 117 2016-04-07T03:42:16.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90510 Milliseconds Vbucket 117 2016-04-07T03:42:16.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90540 Milliseconds Vbucket 117 2016-04-07T03:42:16.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90570 Milliseconds Vbucket 117 2016-04-07T03:42:16.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90600 Milliseconds Vbucket 117 2016-04-07T03:42:16.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90630 Milliseconds Vbucket 117 2016-04-07T03:42:16.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90660 Milliseconds Vbucket 117 2016-04-07T03:42:16.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90690 Milliseconds Vbucket 117 2016-04-07T03:42:16.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90720 Milliseconds Vbucket 117 2016-04-07T03:42:16.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90750 Milliseconds Vbucket 117 2016-04-07T03:42:16.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90780 Milliseconds Vbucket 117 2016-04-07T03:42:16.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90810 Milliseconds Vbucket 117 2016-04-07T03:42:16.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90840 Milliseconds Vbucket 117 2016-04-07T03:42:16.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90870 Milliseconds Vbucket 117 2016-04-07T03:42:16.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90900 Milliseconds Vbucket 117 2016-04-07T03:42:16.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90930 Milliseconds Vbucket 117 2016-04-07T03:42:16.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90960 Milliseconds Vbucket 117 2016-04-07T03:42:16.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90990 Milliseconds Vbucket 117 2016-04-07T03:42:16.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91020 Milliseconds Vbucket 117 2016-04-07T03:42:16.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91050 Milliseconds Vbucket 117 2016-04-07T03:42:16.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91080 Milliseconds Vbucket 117 2016-04-07T03:42:16.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91110 Milliseconds Vbucket 117 2016-04-07T03:42:16.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91140 Milliseconds Vbucket 117 2016-04-07T03:42:16.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91170 Milliseconds Vbucket 117 2016-04-07T03:42:16.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91200 Milliseconds Vbucket 117 2016-04-07T03:42:16.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91230 Milliseconds Vbucket 117 2016-04-07T03:42:16.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91260 Milliseconds Vbucket 117 2016-04-07T03:42:16.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91290 Milliseconds Vbucket 117 2016-04-07T03:42:17.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91320 Milliseconds Vbucket 117 2016-04-07T03:42:17.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91350 Milliseconds Vbucket 117 2016-04-07T03:42:17.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91380 Milliseconds Vbucket 117 2016-04-07T03:42:17.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91410 Milliseconds Vbucket 117 2016-04-07T03:42:17.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91440 Milliseconds Vbucket 117 2016-04-07T03:42:17.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91470 Milliseconds Vbucket 117 2016-04-07T03:42:17.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91500 Milliseconds Vbucket 117 2016-04-07T03:42:17.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91530 Milliseconds Vbucket 117 2016-04-07T03:42:17.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91560 Milliseconds Vbucket 117 2016-04-07T03:42:17.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91590 Milliseconds Vbucket 117 2016-04-07T03:42:17.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91620 Milliseconds Vbucket 117 2016-04-07T03:42:17.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91650 Milliseconds Vbucket 117 2016-04-07T03:42:17.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91680 Milliseconds Vbucket 117 2016-04-07T03:42:17.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91710 Milliseconds Vbucket 117 2016-04-07T03:42:17.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91740 Milliseconds Vbucket 117 2016-04-07T03:42:17.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91770 Milliseconds Vbucket 117 2016-04-07T03:42:17.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91800 Milliseconds Vbucket 117 2016-04-07T03:42:17.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91830 Milliseconds Vbucket 117 2016-04-07T03:42:17.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91860 Milliseconds Vbucket 117 2016-04-07T03:42:17.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91890 Milliseconds Vbucket 117 2016-04-07T03:42:17.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91920 Milliseconds Vbucket 117 2016-04-07T03:42:17.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91950 Milliseconds Vbucket 117 2016-04-07T03:42:17.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91980 Milliseconds Vbucket 117 2016-04-07T03:42:17.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92010 Milliseconds Vbucket 117 2016-04-07T03:42:17.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92040 Milliseconds Vbucket 117 2016-04-07T03:42:17.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92070 Milliseconds Vbucket 117 2016-04-07T03:42:17.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92100 Milliseconds Vbucket 117 2016-04-07T03:42:17.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92130 Milliseconds Vbucket 117 2016-04-07T03:42:17.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92160 Milliseconds Vbucket 117 2016-04-07T03:42:17.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92190 Milliseconds Vbucket 117 2016-04-07T03:42:17.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92220 Milliseconds Vbucket 117 2016-04-07T03:42:17.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92250 Milliseconds Vbucket 117 2016-04-07T03:42:17.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92280 Milliseconds Vbucket 117 2016-04-07T03:42:18.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92310 Milliseconds Vbucket 117 2016-04-07T03:42:18.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92340 Milliseconds Vbucket 117 2016-04-07T03:42:18.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92370 Milliseconds Vbucket 117 2016-04-07T03:42:18.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92400 Milliseconds Vbucket 117 2016-04-07T03:42:18.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92430 Milliseconds Vbucket 117 2016-04-07T03:42:18.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92460 Milliseconds Vbucket 117 2016-04-07T03:42:18.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92490 Milliseconds Vbucket 117 2016-04-07T03:42:18.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92520 Milliseconds Vbucket 117 2016-04-07T03:42:18.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92550 Milliseconds Vbucket 117 2016-04-07T03:42:18.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92580 Milliseconds Vbucket 117 2016-04-07T03:42:18.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92610 Milliseconds Vbucket 117 2016-04-07T03:42:18.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92640 Milliseconds Vbucket 117 2016-04-07T03:42:18.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92670 Milliseconds Vbucket 117 2016-04-07T03:42:18.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92700 Milliseconds Vbucket 117 2016-04-07T03:42:18.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92730 Milliseconds Vbucket 117 2016-04-07T03:42:18.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92760 Milliseconds Vbucket 117 2016-04-07T03:42:18.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92790 Milliseconds Vbucket 117 2016-04-07T03:42:18.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92820 Milliseconds Vbucket 117 2016-04-07T03:42:18.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92850 Milliseconds Vbucket 117 2016-04-07T03:42:18.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92880 Milliseconds Vbucket 117 2016-04-07T03:42:18.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92910 Milliseconds Vbucket 117 2016-04-07T03:42:18.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92940 Milliseconds Vbucket 117 2016-04-07T03:42:18.653-07:00 [Info] CompactionDaemon: Finished compacting index instance:9366861204008862690 2016-04-07T03:42:18.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92970 Milliseconds Vbucket 117 2016-04-07T03:42:18.665-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4490000 QueuedCount 2969 2016-04-07T03:42:18.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93000 Milliseconds Vbucket 117 2016-04-07T03:42:18.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93030 Milliseconds Vbucket 117 2016-04-07T03:42:18.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93060 Milliseconds Vbucket 117 2016-04-07T03:42:18.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93090 Milliseconds Vbucket 117 2016-04-07T03:42:18.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93120 Milliseconds Vbucket 117 2016-04-07T03:42:18.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93150 Milliseconds Vbucket 117 2016-04-07T03:42:18.855-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.057µs CommitTime 50.584119ms TotalFlushTime 2m35.346287342s TotalCommitTime 5.524469408s 2016-04-07T03:42:18.855-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 4492969, 4492969, 27 committed:true 2016-04-07T03:42:18.855-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 4492969, 4492969, 27 committed:true) SnapCreateDur 60.600164ms SnapOpenDur 50.308µs 2016-04-07T03:42:18.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93180 Milliseconds Vbucket 117 2016-04-07T03:42:18.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93210 Milliseconds Vbucket 117 2016-04-07T03:42:18.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93240 Milliseconds Vbucket 117 2016-04-07T03:42:18.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93270 Milliseconds Vbucket 117 2016-04-07T03:42:18.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93300 Milliseconds Vbucket 117 2016-04-07T03:42:19.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93330 Milliseconds Vbucket 117 2016-04-07T03:42:19.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93360 Milliseconds Vbucket 117 2016-04-07T03:42:19.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93390 Milliseconds Vbucket 117 2016-04-07T03:42:19.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93420 Milliseconds Vbucket 117 2016-04-07T03:42:19.126-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4500000 QueuedCount 22395 2016-04-07T03:42:19.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93450 Milliseconds Vbucket 117 2016-04-07T03:42:19.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93480 Milliseconds Vbucket 117 2016-04-07T03:42:19.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93510 Milliseconds Vbucket 117 2016-04-07T03:42:19.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93540 Milliseconds Vbucket 117 2016-04-07T03:42:19.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93570 Milliseconds Vbucket 117 2016-04-07T03:42:19.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93600 Milliseconds Vbucket 117 2016-04-07T03:42:19.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93630 Milliseconds Vbucket 117 2016-04-07T03:42:19.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93660 Milliseconds Vbucket 117 2016-04-07T03:42:19.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93690 Milliseconds Vbucket 117 2016-04-07T03:42:19.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93720 Milliseconds Vbucket 117 2016-04-07T03:42:19.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93750 Milliseconds Vbucket 117 2016-04-07T03:42:19.471-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4510000 QueuedCount 12395 2016-04-07T03:42:19.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93780 Milliseconds Vbucket 117 2016-04-07T03:42:19.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93810 Milliseconds Vbucket 117 2016-04-07T03:42:19.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93840 Milliseconds Vbucket 117 2016-04-07T03:42:19.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93870 Milliseconds Vbucket 117 2016-04-07T03:42:19.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93900 Milliseconds Vbucket 117 2016-04-07T03:42:19.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93930 Milliseconds Vbucket 117 2016-04-07T03:42:19.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93960 Milliseconds Vbucket 117 2016-04-07T03:42:19.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93990 Milliseconds Vbucket 117 2016-04-07T03:42:19.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94020 Milliseconds Vbucket 117 2016-04-07T03:42:19.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94050 Milliseconds Vbucket 117 2016-04-07T03:42:19.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94080 Milliseconds Vbucket 117 2016-04-07T03:42:19.787-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4520000 QueuedCount 2395 2016-04-07T03:42:19.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94110 Milliseconds Vbucket 117 2016-04-07T03:42:19.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94140 Milliseconds Vbucket 117 2016-04-07T03:42:19.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94170 Milliseconds Vbucket 117 2016-04-07T03:42:19.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94200 Milliseconds Vbucket 117 2016-04-07T03:42:19.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94230 Milliseconds Vbucket 117 2016-04-07T03:42:19.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94260 Milliseconds Vbucket 117 2016-04-07T03:42:19.984-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94290 Milliseconds Vbucket 117 2016-04-07T03:42:20.014-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94320 Milliseconds Vbucket 117 2016-04-07T03:42:20.035-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.541µs CommitTime 170.659335ms TotalFlushTime 2m36.301696162s TotalCommitTime 5.695128743s 2016-04-07T03:42:20.035-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 4522395, 4522395, 28 committed:true 2016-04-07T03:42:20.035-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 4522395, 4522395, 28 committed:true) SnapCreateDur 180.835926ms SnapOpenDur 35.504µs 2016-04-07T03:42:20.044-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94350 Milliseconds Vbucket 117 2016-04-07T03:42:20.074-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:20.074-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94380 Milliseconds Vbucket 117 2016-04-07T03:42:20.104-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94410 Milliseconds Vbucket 117 2016-04-07T03:42:20.134-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94440 Milliseconds Vbucket 117 2016-04-07T03:42:20.164-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94470 Milliseconds Vbucket 117 2016-04-07T03:42:20.194-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94500 Milliseconds Vbucket 117 2016-04-07T03:42:20.224-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94530 Milliseconds Vbucket 117 2016-04-07T03:42:20.254-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94560 Milliseconds Vbucket 117 2016-04-07T03:42:20.284-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94590 Milliseconds Vbucket 117 2016-04-07T03:42:20.314-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94620 Milliseconds Vbucket 117 2016-04-07T03:42:20.327-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4530000 QueuedCount 50000 2016-04-07T03:42:20.33-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:20.344-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94650 Milliseconds Vbucket 117 2016-04-07T03:42:20.374-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94680 Milliseconds Vbucket 117 2016-04-07T03:42:20.404-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94710 Milliseconds Vbucket 117 2016-04-07T03:42:20.434-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94740 Milliseconds Vbucket 117 2016-04-07T03:42:20.464-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94770 Milliseconds Vbucket 117 2016-04-07T03:42:20.494-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94800 Milliseconds Vbucket 117 2016-04-07T03:42:20.524-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94830 Milliseconds Vbucket 117 2016-04-07T03:42:20.554-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94860 Milliseconds Vbucket 117 2016-04-07T03:42:20.584-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94890 Milliseconds Vbucket 117 2016-04-07T03:42:20.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94920 Milliseconds Vbucket 117 2016-04-07T03:42:20.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94950 Milliseconds Vbucket 117 2016-04-07T03:42:20.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94980 Milliseconds Vbucket 117 2016-04-07T03:42:20.689-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4540000 QueuedCount 50000 2016-04-07T03:42:20.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95010 Milliseconds Vbucket 117 2016-04-07T03:42:20.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95040 Milliseconds Vbucket 117 2016-04-07T03:42:20.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95070 Milliseconds Vbucket 117 2016-04-07T03:42:20.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95100 Milliseconds Vbucket 117 2016-04-07T03:42:20.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95130 Milliseconds Vbucket 117 2016-04-07T03:42:20.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95160 Milliseconds Vbucket 117 2016-04-07T03:42:20.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95190 Milliseconds Vbucket 117 2016-04-07T03:42:20.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95220 Milliseconds Vbucket 117 2016-04-07T03:42:20.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95250 Milliseconds Vbucket 117 2016-04-07T03:42:20.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95280 Milliseconds Vbucket 117 2016-04-07T03:42:21.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95310 Milliseconds Vbucket 117 2016-04-07T03:42:21.026-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4550000 QueuedCount 50000 2016-04-07T03:42:21.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95340 Milliseconds Vbucket 117 2016-04-07T03:42:21.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95370 Milliseconds Vbucket 117 2016-04-07T03:42:21.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95400 Milliseconds Vbucket 117 2016-04-07T03:42:21.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95430 Milliseconds Vbucket 117 2016-04-07T03:42:21.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95460 Milliseconds Vbucket 117 2016-04-07T03:42:21.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95490 Milliseconds Vbucket 117 2016-04-07T03:42:21.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95520 Milliseconds Vbucket 117 2016-04-07T03:42:21.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95550 Milliseconds Vbucket 117 2016-04-07T03:42:21.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95580 Milliseconds Vbucket 117 2016-04-07T03:42:21.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95610 Milliseconds Vbucket 117 2016-04-07T03:42:21.307-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:21.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95640 Milliseconds Vbucket 117 2016-04-07T03:42:21.361-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4560000 QueuedCount 50000 2016-04-07T03:42:21.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95670 Milliseconds Vbucket 117 2016-04-07T03:42:21.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95700 Milliseconds Vbucket 117 2016-04-07T03:42:21.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95730 Milliseconds Vbucket 117 2016-04-07T03:42:21.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95760 Milliseconds Vbucket 117 2016-04-07T03:42:21.484-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95790 Milliseconds Vbucket 117 2016-04-07T03:42:21.514-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95820 Milliseconds Vbucket 117 2016-04-07T03:42:21.544-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95850 Milliseconds Vbucket 117 2016-04-07T03:42:21.574-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95880 Milliseconds Vbucket 117 2016-04-07T03:42:21.604-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95910 Milliseconds Vbucket 117 2016-04-07T03:42:21.634-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95940 Milliseconds Vbucket 117 2016-04-07T03:42:21.664-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95970 Milliseconds Vbucket 117 2016-04-07T03:42:21.694-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96000 Milliseconds Vbucket 117 2016-04-07T03:42:21.701-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4570000 QueuedCount 50000 2016-04-07T03:42:21.724-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96030 Milliseconds Vbucket 117 2016-04-07T03:42:21.754-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96060 Milliseconds Vbucket 117 2016-04-07T03:42:21.784-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96090 Milliseconds Vbucket 117 2016-04-07T03:42:21.814-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96120 Milliseconds Vbucket 117 2016-04-07T03:42:21.844-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96150 Milliseconds Vbucket 117 2016-04-07T03:42:21.874-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96180 Milliseconds Vbucket 117 2016-04-07T03:42:21.904-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96210 Milliseconds Vbucket 117 2016-04-07T03:42:21.934-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96240 Milliseconds Vbucket 117 2016-04-07T03:42:21.964-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96270 Milliseconds Vbucket 117 2016-04-07T03:42:21.994-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96300 Milliseconds Vbucket 117 2016-04-07T03:42:22.016-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4580000 QueuedCount 50000 2016-04-07T03:42:22.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96330 Milliseconds Vbucket 117 2016-04-07T03:42:22.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96360 Milliseconds Vbucket 117 2016-04-07T03:42:22.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96390 Milliseconds Vbucket 117 2016-04-07T03:42:22.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96420 Milliseconds Vbucket 117 2016-04-07T03:42:22.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96450 Milliseconds Vbucket 117 2016-04-07T03:42:22.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96480 Milliseconds Vbucket 117 2016-04-07T03:42:22.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96510 Milliseconds Vbucket 117 2016-04-07T03:42:22.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96540 Milliseconds Vbucket 117 2016-04-07T03:42:22.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96570 Milliseconds Vbucket 117 2016-04-07T03:42:22.286-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:22.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96600 Milliseconds Vbucket 117 2016-04-07T03:42:22.342-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4590000 QueuedCount 49999 2016-04-07T03:42:22.364-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11330000 2016-04-07T03:42:22.472-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11340000 2016-04-07T03:42:22.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11350000 2016-04-07T03:42:22.549-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11360000 2016-04-07T03:42:22.588-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11370000 2016-04-07T03:42:22.627-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11380000 2016-04-07T03:42:22.631-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4600000 QueuedCount 50000 2016-04-07T03:42:22.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11390000 2016-04-07T03:42:22.708-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11400000 2016-04-07T03:42:22.754-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11410000 2016-04-07T03:42:22.797-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11420000 2016-04-07T03:42:22.838-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11430000 2016-04-07T03:42:22.883-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11440000 2016-04-07T03:42:22.923-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11450000 2016-04-07T03:42:22.968-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11460000 2016-04-07T03:42:22.974-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4610000 QueuedCount 50000 2016-04-07T03:42:23.012-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11470000 2016-04-07T03:42:23.052-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11480000 2016-04-07T03:42:23.1-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11490000 2016-04-07T03:42:23.138-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11500000 2016-04-07T03:42:23.178-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11510000 2016-04-07T03:42:23.219-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11520000 2016-04-07T03:42:23.26-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11530000 2016-04-07T03:42:23.26-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:23.306-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11540000 2016-04-07T03:42:23.326-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4620000 QueuedCount 50000 2016-04-07T03:42:23.349-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11550000 2016-04-07T03:42:23.395-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11560000 2016-04-07T03:42:23.44-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11570000 2016-04-07T03:42:23.481-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11580000 2016-04-07T03:42:23.529-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11590000 2016-04-07T03:42:23.568-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11600000 2016-04-07T03:42:23.611-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11610000 2016-04-07T03:42:23.657-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11620000 2016-04-07T03:42:23.676-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4630000 QueuedCount 50000 2016-04-07T03:42:23.697-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11630000 2016-04-07T03:42:23.742-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11640000 2016-04-07T03:42:23.783-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11650000 2016-04-07T03:42:23.822-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11660000 2016-04-07T03:42:23.866-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11670000 2016-04-07T03:42:23.905-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11680000 2016-04-07T03:42:23.948-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11690000 2016-04-07T03:42:23.99-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11700000 2016-04-07T03:42:24.017-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4640000 QueuedCount 50000 2016-04-07T03:42:24.029-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11710000 2016-04-07T03:42:24.071-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11720000 2016-04-07T03:42:24.108-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11730000 2016-04-07T03:42:24.153-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11740000 2016-04-07T03:42:24.193-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11750000 2016-04-07T03:42:24.233-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11760000 2016-04-07T03:42:24.278-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11770000 2016-04-07T03:42:24.286-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:24.325-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11780000 2016-04-07T03:42:24.359-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4650000 QueuedCount 50000 2016-04-07T03:42:24.367-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11790000 2016-04-07T03:42:24.412-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11800000 2016-04-07T03:42:24.449-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11810000 2016-04-07T03:42:24.494-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11820000 2016-04-07T03:42:24.53-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11830000 2016-04-07T03:42:24.569-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11840000 2016-04-07T03:42:24.608-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11850000 2016-04-07T03:42:24.645-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11860000 2016-04-07T03:42:24.686-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11870000 2016-04-07T03:42:24.697-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4660000 QueuedCount 50000 2016-04-07T03:42:24.723-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11880000 2016-04-07T03:42:24.764-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11890000 2016-04-07T03:42:24.801-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11900000 2016-04-07T03:42:24.837-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11910000 2016-04-07T03:42:24.874-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11920000 2016-04-07T03:42:24.911-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11930000 2016-04-07T03:42:24.948-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11940000 2016-04-07T03:42:24.983-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11950000 2016-04-07T03:42:25.031-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11960000 2016-04-07T03:42:25.035-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4670000 QueuedCount 50000 2016-04-07T03:42:25.068-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11970000 2016-04-07T03:42:25.11-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11980000 2016-04-07T03:42:25.15-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 11990000 2016-04-07T03:42:25.193-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12000000 2016-04-07T03:42:25.238-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12010000 2016-04-07T03:42:25.275-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12020000 2016-04-07T03:42:25.32-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12030000 2016-04-07T03:42:25.343-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4680000 QueuedCount 49332 2016-04-07T03:42:25.362-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12040000 2016-04-07T03:42:25.406-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12050000 2016-04-07T03:42:25.445-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12060000 2016-04-07T03:42:25.482-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12070000 2016-04-07T03:42:25.52-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12080000 2016-04-07T03:42:25.558-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12090000 2016-04-07T03:42:25.595-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12100000 2016-04-07T03:42:25.63-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12110000 2016-04-07T03:42:25.655-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4690000 QueuedCount 39332 2016-04-07T03:42:25.666-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12120000 2016-04-07T03:42:25.702-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12130000 2016-04-07T03:42:25.737-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12140000 2016-04-07T03:42:25.774-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12150000 2016-04-07T03:42:25.812-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12160000 2016-04-07T03:42:25.849-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12170000 2016-04-07T03:42:25.886-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12180000 2016-04-07T03:42:25.922-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12190000 2016-04-07T03:42:25.954-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4700000 QueuedCount 29332 2016-04-07T03:42:25.96-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12200000 2016-04-07T03:42:25.995-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12210000 2016-04-07T03:42:26.032-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12220000 2016-04-07T03:42:26.069-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12230000 2016-04-07T03:42:26.106-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12240000 2016-04-07T03:42:26.142-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12250000 2016-04-07T03:42:26.178-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12260000 2016-04-07T03:42:26.215-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12270000 2016-04-07T03:42:26.251-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12280000 2016-04-07T03:42:26.264-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4710000 QueuedCount 19332 2016-04-07T03:42:26.283-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12290000 2016-04-07T03:42:26.322-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12300000 2016-04-07T03:42:26.359-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12310000 2016-04-07T03:42:26.394-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12320000 2016-04-07T03:42:26.431-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12330000 2016-04-07T03:42:26.465-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12340000 2016-04-07T03:42:26.501-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12350000 2016-04-07T03:42:26.536-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12360000 2016-04-07T03:42:26.56-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4720000 QueuedCount 9332 2016-04-07T03:42:26.572-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12370000 2016-04-07T03:42:26.608-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12380000 2016-04-07T03:42:26.644-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12390000 2016-04-07T03:42:26.681-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12400000 2016-04-07T03:42:26.72-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12410000 2016-04-07T03:42:26.755-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12420000 2016-04-07T03:42:26.791-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12430000 2016-04-07T03:42:26.829-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12440000 2016-04-07T03:42:26.864-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12450000 2016-04-07T03:42:26.903-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12460000 2016-04-07T03:42:26.943-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12470000 2016-04-07T03:42:26.983-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12480000 2016-04-07T03:42:27.02-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12490000 2016-04-07T03:42:27.056-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12500000 2016-04-07T03:42:27.096-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12510000 2016-04-07T03:42:27.135-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12520000 2016-04-07T03:42:27.146-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.52µs CommitTime 279.486399ms TotalFlushTime 2m42.827330798s TotalCommitTime 5.974615142s 2016-04-07T03:42:27.146-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 4729332, 4729332, 29 committed:true 2016-04-07T03:42:27.147-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 4729332, 4729332, 29 committed:true) SnapCreateDur 290.845797ms SnapOpenDur 46.154µs 2016-04-07T03:42:27.179-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12530000 2016-04-07T03:42:27.185-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4730000 QueuedCount 29180 2016-04-07T03:42:27.2-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:27.211-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12540000 2016-04-07T03:42:27.249-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12550000 2016-04-07T03:42:27.283-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12560000 2016-04-07T03:42:27.316-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12570000 2016-04-07T03:42:27.354-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12580000 2016-04-07T03:42:27.389-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12590000 2016-04-07T03:42:27.428-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12600000 2016-04-07T03:42:27.46-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12610000 2016-04-07T03:42:27.5-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12620000 2016-04-07T03:42:27.535-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12630000 2016-04-07T03:42:27.573-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12640000 2016-04-07T03:42:27.607-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4740000 QueuedCount 49970 2016-04-07T03:42:27.611-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12650000 2016-04-07T03:42:27.645-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12660000 2016-04-07T03:42:27.682-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12670000 2016-04-07T03:42:27.716-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12680000 2016-04-07T03:42:27.756-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12690000 2016-04-07T03:42:27.79-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12700000 2016-04-07T03:42:27.828-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12710000 2016-04-07T03:42:27.863-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12720000 2016-04-07T03:42:27.899-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12730000 2016-04-07T03:42:27.936-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12740000 2016-04-07T03:42:27.97-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12750000 2016-04-07T03:42:28.007-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12760000 2016-04-07T03:42:28.008-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4750000 QueuedCount 50000 2016-04-07T03:42:28.042-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12770000 2016-04-07T03:42:28.081-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12780000 2016-04-07T03:42:28.117-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12790000 2016-04-07T03:42:28.157-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12800000 2016-04-07T03:42:28.189-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12810000 2016-04-07T03:42:28.226-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12820000 2016-04-07T03:42:28.263-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12830000 2016-04-07T03:42:28.298-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12840000 2016-04-07T03:42:28.334-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12850000 2016-04-07T03:42:28.347-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4760000 QueuedCount 50000 2016-04-07T03:42:28.37-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12860000 2016-04-07T03:42:28.409-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12870000 2016-04-07T03:42:28.444-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12880000 2016-04-07T03:42:28.479-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12890000 2016-04-07T03:42:28.514-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12900000 2016-04-07T03:42:28.55-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:28.551-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12910000 2016-04-07T03:42:28.589-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12920000 2016-04-07T03:42:28.625-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12930000 2016-04-07T03:42:28.661-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12940000 2016-04-07T03:42:28.695-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12950000 2016-04-07T03:42:28.699-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4770000 QueuedCount 50000 2016-04-07T03:42:28.731-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12960000 2016-04-07T03:42:28.766-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12970000 2016-04-07T03:42:28.804-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12980000 2016-04-07T03:42:28.839-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 12990000 2016-04-07T03:42:28.873-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13000000 2016-04-07T03:42:28.91-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13010000 2016-04-07T03:42:28.947-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13020000 2016-04-07T03:42:28.981-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13030000 2016-04-07T03:42:29.021-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13040000 2016-04-07T03:42:29.046-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4780000 QueuedCount 50000 2016-04-07T03:42:29.055-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13050000 2016-04-07T03:42:29.095-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13060000 2016-04-07T03:42:29.129-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13070000 2016-04-07T03:42:29.17-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13080000 2016-04-07T03:42:29.202-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13090000 2016-04-07T03:42:29.243-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13100000 2016-04-07T03:42:29.279-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13110000 2016-04-07T03:42:29.319-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13120000 2016-04-07T03:42:29.357-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13130000 2016-04-07T03:42:29.393-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4790000 QueuedCount 50000 2016-04-07T03:42:29.394-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13140000 2016-04-07T03:42:29.432-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13150000 2016-04-07T03:42:29.468-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13160000 2016-04-07T03:42:29.507-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13170000 2016-04-07T03:42:29.543-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13180000 2016-04-07T03:42:29.586-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13190000 2016-04-07T03:42:29.623-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13200000 2016-04-07T03:42:29.665-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13210000 2016-04-07T03:42:29.7-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13220000 2016-04-07T03:42:29.737-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4800000 QueuedCount 50000 2016-04-07T03:42:29.74-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13230000 2016-04-07T03:42:29.748-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:29.778-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13240000 2016-04-07T03:42:29.813-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13250000 2016-04-07T03:42:29.852-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13260000 2016-04-07T03:42:29.885-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13270000 2016-04-07T03:42:29.928-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13280000 2016-04-07T03:42:29.962-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13290000 2016-04-07T03:42:30.002-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13300000 2016-04-07T03:42:30.04-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13310000 2016-04-07T03:42:30.079-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13320000 2016-04-07T03:42:30.114-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4810000 QueuedCount 50000 2016-04-07T03:42:30.118-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13330000 2016-04-07T03:42:30.157-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13340000 2016-04-07T03:42:30.195-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13350000 2016-04-07T03:42:30.241-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13360000 2016-04-07T03:42:30.284-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13370000 2016-04-07T03:42:30.332-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13380000 2016-04-07T03:42:30.374-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13390000 2016-04-07T03:42:30.422-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13400000 2016-04-07T03:42:30.465-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13410000 2016-04-07T03:42:30.51-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4820000 QueuedCount 49998 2016-04-07T03:42:30.513-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13420000 2016-04-07T03:42:30.557-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13430000 2016-04-07T03:42:30.603-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13440000 2016-04-07T03:42:30.653-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13450000 2016-04-07T03:42:30.7-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13460000 2016-04-07T03:42:30.75-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13470000 2016-04-07T03:42:30.793-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13480000 2016-04-07T03:42:30.842-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13490000 2016-04-07T03:42:30.887-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13500000 2016-04-07T03:42:30.907-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4830000 QueuedCount 50000 2016-04-07T03:42:30.934-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13510000 2016-04-07T03:42:30.979-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13520000 2016-04-07T03:42:31.025-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13530000 2016-04-07T03:42:31.069-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13540000 2016-04-07T03:42:31.12-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13550000 2016-04-07T03:42:31.169-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13560000 2016-04-07T03:42:31.225-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13570000 2016-04-07T03:42:31.276-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4840000 QueuedCount 46640 2016-04-07T03:42:31.282-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13580000 2016-04-07T03:42:31.336-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13590000 2016-04-07T03:42:31.39-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13600000 2016-04-07T03:42:31.442-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13610000 2016-04-07T03:42:31.495-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13620000 2016-04-07T03:42:31.546-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13630000 2016-04-07T03:42:31.597-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4850000 QueuedCount 36640 2016-04-07T03:42:31.602-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13640000 2016-04-07T03:42:31.659-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13650000 2016-04-07T03:42:31.711-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13660000 2016-04-07T03:42:31.766-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13670000 2016-04-07T03:42:31.817-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13680000 2016-04-07T03:42:31.872-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13690000 2016-04-07T03:42:31.927-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13700000 2016-04-07T03:42:31.98-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4860000 QueuedCount 26640 2016-04-07T03:42:31.98-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13710000 2016-04-07T03:42:32.323-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4870000 QueuedCount 16640 2016-04-07T03:42:32.663-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4880000 QueuedCount 6640 2016-04-07T03:42:33.216-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.826µs CommitTime 318.657666ms TotalFlushTime 2m48.388615815s TotalCommitTime 6.293272808s 2016-04-07T03:42:33.216-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 4886640, 4886640, 30 committed:true 2016-04-07T03:42:33.216-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 4886640, 4886640, 30 committed:true) SnapCreateDur 329.893032ms SnapOpenDur 40.41µs 2016-04-07T03:42:33.254-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:33.278-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13720000 2016-04-07T03:42:33.314-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13730000 2016-04-07T03:42:33.33-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:33.339-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4890000 QueuedCount 49102 2016-04-07T03:42:33.35-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13740000 2016-04-07T03:42:33.389-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13750000 2016-04-07T03:42:33.425-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13760000 2016-04-07T03:42:33.979-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4900000 QueuedCount 50000 2016-04-07T03:42:34.083-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13770000 2016-04-07T03:42:34.416-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4910000 QueuedCount 50000 2016-04-07T03:42:34.424-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13780000 2016-04-07T03:42:34.819-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4920000 QueuedCount 50000 2016-04-07T03:42:34.821-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13790000 2016-04-07T03:42:34.98-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:35.188-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4930000 QueuedCount 50000 2016-04-07T03:42:35.281-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13800000 2016-04-07T03:42:35.555-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4940000 QueuedCount 50000 2016-04-07T03:42:35.908-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4950000 QueuedCount 50000 2016-04-07T03:42:35.917-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13810000 2016-04-07T03:42:36.269-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4960000 QueuedCount 50000 2016-04-07T03:42:36.586-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4970000 QueuedCount 42824 2016-04-07T03:42:36.891-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4980000 QueuedCount 32824 2016-04-07T03:42:37.215-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 4990000 QueuedCount 22824 2016-04-07T03:42:37.524-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5000000 QueuedCount 12824 2016-04-07T03:42:37.819-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5010000 QueuedCount 2824 2016-04-07T03:42:38.198-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.957µs CommitTime 296.786198ms TotalFlushTime 2m52.909873221s TotalCommitTime 6.590059006s 2016-04-07T03:42:38.198-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 5012824, 5012824, 31 committed:true 2016-04-07T03:42:38.198-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 5012824, 5012824, 31 committed:true) SnapCreateDur 306.637816ms SnapOpenDur 53.478µs 2016-04-07T03:42:38.235-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13820000 2016-04-07T03:42:38.241-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:38.278-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13830000 2016-04-07T03:42:38.325-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13840000 2016-04-07T03:42:38.365-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13850000 2016-04-07T03:42:38.414-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13860000 2016-04-07T03:42:38.462-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13870000 2016-04-07T03:42:38.497-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5020000 QueuedCount 50000 2016-04-07T03:42:38.51-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13880000 2016-04-07T03:42:38.577-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13890000 2016-04-07T03:42:38.645-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13900000 2016-04-07T03:42:38.78-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13910000 2016-04-07T03:42:38.953-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5030000 QueuedCount 50000 2016-04-07T03:42:39.05-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13920000 2016-04-07T03:42:39.185-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:39.265-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13930000 2016-04-07T03:42:39.331-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5040000 QueuedCount 50000 2016-04-07T03:42:39.661-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13940000 2016-04-07T03:42:39.692-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5050000 QueuedCount 50000 2016-04-07T03:42:39.941-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13950000 2016-04-07T03:42:40.05-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5060000 QueuedCount 50000 2016-04-07T03:42:40.279-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13960000 2016-04-07T03:42:40.353-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5070000 QueuedCount 50000 2016-04-07T03:42:40.48-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:40.681-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5080000 QueuedCount 50000 2016-04-07T03:42:40.954-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13970000 2016-04-07T03:42:41.033-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5090000 QueuedCount 50000 2016-04-07T03:42:41.389-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5100000 QueuedCount 50000 2016-04-07T03:42:41.416-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13980000 2016-04-07T03:42:41.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 13990000 2016-04-07T03:42:41.728-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5110000 QueuedCount 49140 2016-04-07T03:42:42.026-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5120000 QueuedCount 39140 2016-04-07T03:42:42.342-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5130000 QueuedCount 29140 2016-04-07T03:42:42.646-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5140000 QueuedCount 19140 2016-04-07T03:42:42.914-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5150000 QueuedCount 9140 2016-04-07T03:42:43.558-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.062µs CommitTime 316.916449ms TotalFlushTime 2m57.694851827s TotalCommitTime 6.906975455s 2016-04-07T03:42:43.558-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 5159140, 5159140, 32 committed:true 2016-04-07T03:42:43.558-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 5159140, 5159140, 32 committed:true) SnapCreateDur 328.254226ms SnapOpenDur 58.786µs 2016-04-07T03:42:43.596-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5160000 QueuedCount 27326 2016-04-07T03:42:43.605-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:43.636-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14000000 2016-04-07T03:42:43.68-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14010000 2016-04-07T03:42:43.724-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14020000 2016-04-07T03:42:43.77-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14030000 2016-04-07T03:42:43.993-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5170000 QueuedCount 50000 2016-04-07T03:42:44.086-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14040000 2016-04-07T03:42:44.384-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5180000 QueuedCount 50000 2016-04-07T03:42:44.428-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14050000 2016-04-07T03:42:44.541-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:44.75-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5190000 QueuedCount 50000 2016-04-07T03:42:44.923-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14060000 2016-04-07T03:42:45.063-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5200000 QueuedCount 50000 2016-04-07T03:42:45.427-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5210000 QueuedCount 50000 2016-04-07T03:42:45.445-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14070000 2016-04-07T03:42:45.675-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14080000 2016-04-07T03:42:45.781-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5220000 QueuedCount 50000 2016-04-07T03:42:45.836-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:46.104-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5230000 QueuedCount 50000 2016-04-07T03:42:46.165-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14090000 2016-04-07T03:42:46.447-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5240000 QueuedCount 50000 2016-04-07T03:42:46.621-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14100000 2016-04-07T03:42:46.799-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5250000 QueuedCount 50000 2016-04-07T03:42:46.901-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14110000 2016-04-07T03:42:47.031-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14120000 2016-04-07T03:42:47.159-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5260000 QueuedCount 48687 2016-04-07T03:42:47.484-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5270000 QueuedCount 38687 2016-04-07T03:42:47.748-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5280000 QueuedCount 28687 2016-04-07T03:42:48.058-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5290000 QueuedCount 18687 2016-04-07T03:42:48.378-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5300000 QueuedCount 8687 2016-04-07T03:42:48.653-07:00 [Info] CompactionDaemon: Checking fragmentation, IndexInstId: 9366861204008862690 Data:1688175879, Disk:5123448832, ExtraSnapshotData:0, Fragmentation:67.05000997656104% 2016-04-07T03:42:48.653-07:00 [Info] CompactionDaemon: Compacting index instance:9366861204008862690 2016-04-07T03:42:48.682-07:00 [Info] ForestDBSlice::Compact Compacting upto SeqNum 4522395. Slice Id 0, IndexInstId 9366861204008862690, IndexDefnId 9366861204008862690 2016-04-07T03:42:49.021-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.015µs CommitTime 236.472µs TotalFlushTime 3m2.565240584s TotalCommitTime 6.907211927s 2016-04-07T03:42:49.022-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 5308687, 5308687, 33 committed:true 2016-04-07T03:42:49.022-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 5308687, 5308687, 33 committed:true) SnapCreateDur 343.602758ms SnapOpenDur 48.278µs 2016-04-07T03:42:49.066-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:49.067-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5310000 QueuedCount 36040 2016-04-07T03:42:49.09-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14130000 2016-04-07T03:42:49.137-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14140000 2016-04-07T03:42:49.179-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14150000 2016-04-07T03:42:49.228-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14160000 2016-04-07T03:42:49.27-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14170000 2016-04-07T03:42:49.525-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5320000 QueuedCount 50000 2016-04-07T03:42:49.703-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14180000 2016-04-07T03:42:49.961-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5330000 QueuedCount 50000 2016-04-07T03:42:50.125-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:50.168-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14190000 2016-04-07T03:42:50.377-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5340000 QueuedCount 50000 2016-04-07T03:42:50.63-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14200000 2016-04-07T03:42:50.755-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5350000 QueuedCount 50000 2016-04-07T03:42:50.943-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14210000 2016-04-07T03:42:50.994-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14220000 2016-04-07T03:42:51.041-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14230000 2016-04-07T03:42:51.147-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5360000 QueuedCount 50000 2016-04-07T03:42:51.245-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14240000 2016-04-07T03:42:51.467-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5370000 QueuedCount 50000 2016-04-07T03:42:51.535-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:51.613-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14250000 2016-04-07T03:42:51.828-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5380000 QueuedCount 50000 2016-04-07T03:42:52.11-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14260000 2016-04-07T03:42:52.176-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5390000 QueuedCount 50000 2016-04-07T03:42:52.392-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14270000 2016-04-07T03:42:52.536-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5400000 QueuedCount 50000 2016-04-07T03:42:52.642-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14280000 2016-04-07T03:42:52.893-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5410000 QueuedCount 46354 2016-04-07T03:42:53.222-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5420000 QueuedCount 36354 2016-04-07T03:42:53.531-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5430000 QueuedCount 26354 2016-04-07T03:42:53.84-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5440000 QueuedCount 16354 2016-04-07T03:42:54.116-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5450000 QueuedCount 6354 2016-04-07T03:42:54.669-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.38µs CommitTime 315.810645ms TotalFlushTime 3m7.642613434s TotalCommitTime 7.223022572s 2016-04-07T03:42:54.669-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 5456354, 5456354, 34 committed:true 2016-04-07T03:42:54.669-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 5456354, 5456354, 34 committed:true) SnapCreateDur 326.981682ms SnapOpenDur 52.586µs 2016-04-07T03:42:54.712-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:54.81-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5460000 QueuedCount 50000 2016-04-07T03:42:55.207-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5470000 QueuedCount 50000 2016-04-07T03:42:55.425-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:55.584-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5480000 QueuedCount 50000 2016-04-07T03:42:55.962-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5490000 QueuedCount 50000 2016-04-07T03:42:56.328-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5500000 QueuedCount 50000 2016-04-07T03:42:56.626-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:42:56.683-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5510000 QueuedCount 50000 2016-04-07T03:42:57.062-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5520000 QueuedCount 50000 2016-04-07T03:42:58.093-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5530000 QueuedCount 50000 2016-04-07T03:42:58.464-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5540000 QueuedCount 50000 2016-04-07T03:42:58.803-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5550000 QueuedCount 42927 2016-04-07T03:42:59.142-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5560000 QueuedCount 32927 2016-04-07T03:42:59.48-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5570000 QueuedCount 22927 2016-04-07T03:42:59.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 117 2016-04-07T03:42:59.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 117 2016-04-07T03:42:59.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 117 2016-04-07T03:42:59.79-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5580000 QueuedCount 12927 2016-04-07T03:42:59.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 117 2016-04-07T03:42:59.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 117 2016-04-07T03:42:59.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 117 2016-04-07T03:42:59.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 117 2016-04-07T03:42:59.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 117 2016-04-07T03:42:59.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 117 2016-04-07T03:42:59.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 117 2016-04-07T03:43:00.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 117 2016-04-07T03:43:00.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 117 2016-04-07T03:43:00.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 117 2016-04-07T03:43:00.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 117 2016-04-07T03:43:00.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 117 2016-04-07T03:43:00.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 117 2016-04-07T03:43:00.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 117 2016-04-07T03:43:00.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 117 2016-04-07T03:43:00.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 117 2016-04-07T03:43:00.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 117 2016-04-07T03:43:00.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 117 2016-04-07T03:43:00.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 117 2016-04-07T03:43:00.36-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5590000 QueuedCount 2927 2016-04-07T03:43:00.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 117 2016-04-07T03:43:00.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 117 2016-04-07T03:43:00.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 117 2016-04-07T03:43:00.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 117 2016-04-07T03:43:00.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 117 2016-04-07T03:43:00.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 117 2016-04-07T03:43:00.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 117 2016-04-07T03:43:00.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 117 2016-04-07T03:43:00.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 117 2016-04-07T03:43:00.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 117 2016-04-07T03:43:00.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 117 2016-04-07T03:43:00.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 117 2016-04-07T03:43:00.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 117 2016-04-07T03:43:00.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 117 2016-04-07T03:43:00.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 117 2016-04-07T03:43:00.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 117 2016-04-07T03:43:00.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 117 2016-04-07T03:43:00.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 117 2016-04-07T03:43:00.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 117 2016-04-07T03:43:00.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 117 2016-04-07T03:43:00.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 117 2016-04-07T03:43:01.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 117 2016-04-07T03:43:01.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 117 2016-04-07T03:43:01.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 117 2016-04-07T03:43:01.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 117 2016-04-07T03:43:01.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 117 2016-04-07T03:43:01.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 117 2016-04-07T03:43:01.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 117 2016-04-07T03:43:01.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 117 2016-04-07T03:43:01.245-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.694µs CommitTime 363.989498ms TotalFlushTime 3m13.626635287s TotalCommitTime 7.58701207s 2016-04-07T03:43:01.245-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 5592927, 5592927, 35 committed:true 2016-04-07T03:43:01.245-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 5592927, 5592927, 35 committed:true) SnapCreateDur 373.705383ms SnapOpenDur 37.797µs 2016-04-07T03:43:01.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 117 2016-04-07T03:43:01.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 117 2016-04-07T03:43:01.289-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:01.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 117 2016-04-07T03:43:01.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 117 2016-04-07T03:43:01.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 117 2016-04-07T03:43:01.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 117 2016-04-07T03:43:01.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 117 2016-04-07T03:43:01.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 117 2016-04-07T03:43:01.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 117 2016-04-07T03:43:01.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 117 2016-04-07T03:43:01.544-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5600000 QueuedCount 50000 2016-04-07T03:43:01.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 117 2016-04-07T03:43:01.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 117 2016-04-07T03:43:01.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 117 2016-04-07T03:43:01.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 117 2016-04-07T03:43:01.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 117 2016-04-07T03:43:01.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 117 2016-04-07T03:43:01.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 117 2016-04-07T03:43:01.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 117 2016-04-07T03:43:01.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 117 2016-04-07T03:43:01.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 117 2016-04-07T03:43:01.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 117 2016-04-07T03:43:01.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 117 2016-04-07T03:43:01.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 117 2016-04-07T03:43:01.931-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5610000 QueuedCount 50000 2016-04-07T03:43:01.94-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:01.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 117 2016-04-07T03:43:01.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 117 2016-04-07T03:43:02.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 117 2016-04-07T03:43:02.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 117 2016-04-07T03:43:02.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 117 2016-04-07T03:43:02.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 117 2016-04-07T03:43:02.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 117 2016-04-07T03:43:02.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 117 2016-04-07T03:43:02.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 117 2016-04-07T03:43:02.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 117 2016-04-07T03:43:02.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 117 2016-04-07T03:43:02.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 117 2016-04-07T03:43:02.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 117 2016-04-07T03:43:02.321-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5620000 QueuedCount 50000 2016-04-07T03:43:02.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 117 2016-04-07T03:43:02.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 117 2016-04-07T03:43:02.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 117 2016-04-07T03:43:02.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 117 2016-04-07T03:43:02.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 117 2016-04-07T03:43:02.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 117 2016-04-07T03:43:02.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 117 2016-04-07T03:43:02.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 117 2016-04-07T03:43:02.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 117 2016-04-07T03:43:02.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 117 2016-04-07T03:43:02.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 117 2016-04-07T03:43:02.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 117 2016-04-07T03:43:02.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 117 2016-04-07T03:43:02.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 117 2016-04-07T03:43:02.738-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5630000 QueuedCount 50000 2016-04-07T03:43:02.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 117 2016-04-07T03:43:02.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 117 2016-04-07T03:43:02.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 117 2016-04-07T03:43:02.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 117 2016-04-07T03:43:02.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 117 2016-04-07T03:43:02.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 117 2016-04-07T03:43:02.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 117 2016-04-07T03:43:02.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 117 2016-04-07T03:43:02.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 117 2016-04-07T03:43:03.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 117 2016-04-07T03:43:03.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 117 2016-04-07T03:43:03.085-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5640000 QueuedCount 50000 2016-04-07T03:43:03.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 117 2016-04-07T03:43:03.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 117 2016-04-07T03:43:03.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 117 2016-04-07T03:43:03.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 117 2016-04-07T03:43:03.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 117 2016-04-07T03:43:03.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 117 2016-04-07T03:43:03.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 117 2016-04-07T03:43:03.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 117 2016-04-07T03:43:03.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 117 2016-04-07T03:43:03.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 117 2016-04-07T03:43:03.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 117 2016-04-07T03:43:03.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8700 Milliseconds Vbucket 117 2016-04-07T03:43:03.433-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5650000 QueuedCount 44398 2016-04-07T03:43:03.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8730 Milliseconds Vbucket 117 2016-04-07T03:43:03.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8760 Milliseconds Vbucket 117 2016-04-07T03:43:03.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8790 Milliseconds Vbucket 117 2016-04-07T03:43:03.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8820 Milliseconds Vbucket 117 2016-04-07T03:43:03.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8850 Milliseconds Vbucket 117 2016-04-07T03:43:03.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8880 Milliseconds Vbucket 117 2016-04-07T03:43:03.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8910 Milliseconds Vbucket 117 2016-04-07T03:43:03.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8940 Milliseconds Vbucket 117 2016-04-07T03:43:03.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8970 Milliseconds Vbucket 117 2016-04-07T03:43:03.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9000 Milliseconds Vbucket 117 2016-04-07T03:43:03.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9030 Milliseconds Vbucket 117 2016-04-07T03:43:03.752-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5660000 QueuedCount 34398 2016-04-07T03:43:03.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9060 Milliseconds Vbucket 117 2016-04-07T03:43:03.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9090 Milliseconds Vbucket 117 2016-04-07T03:43:03.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9120 Milliseconds Vbucket 117 2016-04-07T03:43:03.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9150 Milliseconds Vbucket 117 2016-04-07T03:43:03.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9180 Milliseconds Vbucket 117 2016-04-07T03:43:03.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9210 Milliseconds Vbucket 117 2016-04-07T03:43:03.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9240 Milliseconds Vbucket 117 2016-04-07T03:43:03.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9270 Milliseconds Vbucket 117 2016-04-07T03:43:04.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9300 Milliseconds Vbucket 117 2016-04-07T03:43:04.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9330 Milliseconds Vbucket 117 2016-04-07T03:43:04.07-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5670000 QueuedCount 24398 2016-04-07T03:43:04.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9360 Milliseconds Vbucket 117 2016-04-07T03:43:04.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9390 Milliseconds Vbucket 117 2016-04-07T03:43:04.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9420 Milliseconds Vbucket 117 2016-04-07T03:43:04.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9450 Milliseconds Vbucket 117 2016-04-07T03:43:04.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9480 Milliseconds Vbucket 117 2016-04-07T03:43:04.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9510 Milliseconds Vbucket 117 2016-04-07T03:43:04.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9540 Milliseconds Vbucket 117 2016-04-07T03:43:04.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9570 Milliseconds Vbucket 117 2016-04-07T03:43:04.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9600 Milliseconds Vbucket 117 2016-04-07T03:43:04.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9630 Milliseconds Vbucket 117 2016-04-07T03:43:04.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9660 Milliseconds Vbucket 117 2016-04-07T03:43:04.379-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5680000 QueuedCount 14398 2016-04-07T03:43:04.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9690 Milliseconds Vbucket 117 2016-04-07T03:43:04.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9720 Milliseconds Vbucket 117 2016-04-07T03:43:04.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9750 Milliseconds Vbucket 117 2016-04-07T03:43:04.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9780 Milliseconds Vbucket 117 2016-04-07T03:43:04.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9810 Milliseconds Vbucket 117 2016-04-07T03:43:04.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9840 Milliseconds Vbucket 117 2016-04-07T03:43:04.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9870 Milliseconds Vbucket 117 2016-04-07T03:43:04.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9900 Milliseconds Vbucket 117 2016-04-07T03:43:04.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9930 Milliseconds Vbucket 117 2016-04-07T03:43:04.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9960 Milliseconds Vbucket 117 2016-04-07T03:43:04.699-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5690000 QueuedCount 4398 2016-04-07T03:43:04.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9990 Milliseconds Vbucket 117 2016-04-07T03:43:04.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10020 Milliseconds Vbucket 117 2016-04-07T03:43:04.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10050 Milliseconds Vbucket 117 2016-04-07T03:43:04.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10080 Milliseconds Vbucket 117 2016-04-07T03:43:04.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10110 Milliseconds Vbucket 117 2016-04-07T03:43:04.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10140 Milliseconds Vbucket 117 2016-04-07T03:43:04.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10170 Milliseconds Vbucket 117 2016-04-07T03:43:04.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10200 Milliseconds Vbucket 117 2016-04-07T03:43:04.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10230 Milliseconds Vbucket 117 2016-04-07T03:43:04.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10260 Milliseconds Vbucket 117 2016-04-07T03:43:05.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10290 Milliseconds Vbucket 117 2016-04-07T03:43:05.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10320 Milliseconds Vbucket 117 2016-04-07T03:43:05.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10350 Milliseconds Vbucket 117 2016-04-07T03:43:05.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10380 Milliseconds Vbucket 117 2016-04-07T03:43:05.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10410 Milliseconds Vbucket 117 2016-04-07T03:43:05.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10440 Milliseconds Vbucket 117 2016-04-07T03:43:05.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10470 Milliseconds Vbucket 117 2016-04-07T03:43:05.19-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.22µs CommitTime 348.632595ms TotalFlushTime 3m17.0458714s TotalCommitTime 7.935644665s 2016-04-07T03:43:05.19-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 5694398, 5694398, 36 committed:true 2016-04-07T03:43:05.19-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 5694398, 5694398, 36 committed:true) SnapCreateDur 358.685879ms SnapOpenDur 38.922µs 2016-04-07T03:43:05.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10500 Milliseconds Vbucket 117 2016-04-07T03:43:05.243-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:05.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10530 Milliseconds Vbucket 117 2016-04-07T03:43:05.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10560 Milliseconds Vbucket 117 2016-04-07T03:43:05.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10590 Milliseconds Vbucket 117 2016-04-07T03:43:05.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10620 Milliseconds Vbucket 117 2016-04-07T03:43:05.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10650 Milliseconds Vbucket 117 2016-04-07T03:43:05.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10680 Milliseconds Vbucket 117 2016-04-07T03:43:05.414-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5700000 QueuedCount 50000 2016-04-07T03:43:05.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10710 Milliseconds Vbucket 117 2016-04-07T03:43:05.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10740 Milliseconds Vbucket 117 2016-04-07T03:43:05.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10770 Milliseconds Vbucket 117 2016-04-07T03:43:05.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10800 Milliseconds Vbucket 117 2016-04-07T03:43:05.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10830 Milliseconds Vbucket 117 2016-04-07T03:43:05.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10860 Milliseconds Vbucket 117 2016-04-07T03:43:05.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10890 Milliseconds Vbucket 117 2016-04-07T03:43:05.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10920 Milliseconds Vbucket 117 2016-04-07T03:43:05.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10950 Milliseconds Vbucket 117 2016-04-07T03:43:05.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10980 Milliseconds Vbucket 117 2016-04-07T03:43:05.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11010 Milliseconds Vbucket 117 2016-04-07T03:43:05.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11040 Milliseconds Vbucket 117 2016-04-07T03:43:05.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11070 Milliseconds Vbucket 117 2016-04-07T03:43:05.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11100 Milliseconds Vbucket 117 2016-04-07T03:43:05.825-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5710000 QueuedCount 50000 2016-04-07T03:43:05.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11130 Milliseconds Vbucket 117 2016-04-07T03:43:05.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11160 Milliseconds Vbucket 117 2016-04-07T03:43:05.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11190 Milliseconds Vbucket 117 2016-04-07T03:43:05.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11220 Milliseconds Vbucket 117 2016-04-07T03:43:05.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11250 Milliseconds Vbucket 117 2016-04-07T03:43:05.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11280 Milliseconds Vbucket 117 2016-04-07T03:43:06.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11310 Milliseconds Vbucket 117 2016-04-07T03:43:06.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11340 Milliseconds Vbucket 117 2016-04-07T03:43:06.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11370 Milliseconds Vbucket 117 2016-04-07T03:43:06.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11400 Milliseconds Vbucket 117 2016-04-07T03:43:06.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11430 Milliseconds Vbucket 117 2016-04-07T03:43:06.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11460 Milliseconds Vbucket 117 2016-04-07T03:43:06.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11490 Milliseconds Vbucket 117 2016-04-07T03:43:06.233-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5720000 QueuedCount 50000 2016-04-07T03:43:06.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11520 Milliseconds Vbucket 117 2016-04-07T03:43:06.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11550 Milliseconds Vbucket 117 2016-04-07T03:43:06.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11580 Milliseconds Vbucket 117 2016-04-07T03:43:06.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11610 Milliseconds Vbucket 117 2016-04-07T03:43:06.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11640 Milliseconds Vbucket 117 2016-04-07T03:43:06.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11670 Milliseconds Vbucket 117 2016-04-07T03:43:06.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11700 Milliseconds Vbucket 117 2016-04-07T03:43:06.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11730 Milliseconds Vbucket 117 2016-04-07T03:43:06.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11760 Milliseconds Vbucket 117 2016-04-07T03:43:06.482-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:06.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11790 Milliseconds Vbucket 117 2016-04-07T03:43:06.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11820 Milliseconds Vbucket 117 2016-04-07T03:43:06.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11850 Milliseconds Vbucket 117 2016-04-07T03:43:06.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11880 Milliseconds Vbucket 117 2016-04-07T03:43:06.616-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5730000 QueuedCount 50000 2016-04-07T03:43:06.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11910 Milliseconds Vbucket 117 2016-04-07T03:43:06.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11940 Milliseconds Vbucket 117 2016-04-07T03:43:06.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11970 Milliseconds Vbucket 117 2016-04-07T03:43:06.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12000 Milliseconds Vbucket 117 2016-04-07T03:43:06.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12030 Milliseconds Vbucket 117 2016-04-07T03:43:06.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12060 Milliseconds Vbucket 117 2016-04-07T03:43:06.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12090 Milliseconds Vbucket 117 2016-04-07T03:43:06.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12120 Milliseconds Vbucket 117 2016-04-07T03:43:06.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12150 Milliseconds Vbucket 117 2016-04-07T03:43:06.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12180 Milliseconds Vbucket 117 2016-04-07T03:43:06.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12210 Milliseconds Vbucket 117 2016-04-07T03:43:06.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12240 Milliseconds Vbucket 117 2016-04-07T03:43:06.978-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5740000 QueuedCount 50000 2016-04-07T03:43:06.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12270 Milliseconds Vbucket 117 2016-04-07T03:43:07.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12300 Milliseconds Vbucket 117 2016-04-07T03:43:07.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12330 Milliseconds Vbucket 117 2016-04-07T03:43:07.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12360 Milliseconds Vbucket 117 2016-04-07T03:43:07.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12390 Milliseconds Vbucket 117 2016-04-07T03:43:07.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12420 Milliseconds Vbucket 117 2016-04-07T03:43:07.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12450 Milliseconds Vbucket 117 2016-04-07T03:43:07.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12480 Milliseconds Vbucket 117 2016-04-07T03:43:07.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12510 Milliseconds Vbucket 117 2016-04-07T03:43:07.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12540 Milliseconds Vbucket 117 2016-04-07T03:43:07.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12570 Milliseconds Vbucket 117 2016-04-07T03:43:07.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12600 Milliseconds Vbucket 117 2016-04-07T03:43:07.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12630 Milliseconds Vbucket 117 2016-04-07T03:43:07.348-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5750000 QueuedCount 50000 2016-04-07T03:43:07.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12660 Milliseconds Vbucket 117 2016-04-07T03:43:07.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12690 Milliseconds Vbucket 117 2016-04-07T03:43:07.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12720 Milliseconds Vbucket 117 2016-04-07T03:43:07.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12750 Milliseconds Vbucket 117 2016-04-07T03:43:07.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12780 Milliseconds Vbucket 117 2016-04-07T03:43:07.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12810 Milliseconds Vbucket 117 2016-04-07T03:43:07.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12840 Milliseconds Vbucket 117 2016-04-07T03:43:07.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12870 Milliseconds Vbucket 117 2016-04-07T03:43:07.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12900 Milliseconds Vbucket 117 2016-04-07T03:43:07.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12930 Milliseconds Vbucket 117 2016-04-07T03:43:07.666-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:07.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12960 Milliseconds Vbucket 117 2016-04-07T03:43:07.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12990 Milliseconds Vbucket 117 2016-04-07T03:43:07.716-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5760000 QueuedCount 50000 2016-04-07T03:43:07.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13020 Milliseconds Vbucket 117 2016-04-07T03:43:07.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13050 Milliseconds Vbucket 117 2016-04-07T03:43:07.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13080 Milliseconds Vbucket 117 2016-04-07T03:43:07.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13110 Milliseconds Vbucket 117 2016-04-07T03:43:07.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13140 Milliseconds Vbucket 117 2016-04-07T03:43:07.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13170 Milliseconds Vbucket 117 2016-04-07T03:43:07.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13200 Milliseconds Vbucket 117 2016-04-07T03:43:07.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13230 Milliseconds Vbucket 117 2016-04-07T03:43:07.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13260 Milliseconds Vbucket 117 2016-04-07T03:43:08.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13290 Milliseconds Vbucket 117 2016-04-07T03:43:08.03-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5770000 QueuedCount 50000 2016-04-07T03:43:08.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13320 Milliseconds Vbucket 117 2016-04-07T03:43:08.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13350 Milliseconds Vbucket 117 2016-04-07T03:43:08.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13380 Milliseconds Vbucket 117 2016-04-07T03:43:08.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13410 Milliseconds Vbucket 117 2016-04-07T03:43:08.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13440 Milliseconds Vbucket 117 2016-04-07T03:43:08.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13470 Milliseconds Vbucket 117 2016-04-07T03:43:08.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13500 Milliseconds Vbucket 117 2016-04-07T03:43:08.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13530 Milliseconds Vbucket 117 2016-04-07T03:43:08.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13560 Milliseconds Vbucket 117 2016-04-07T03:43:08.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13590 Milliseconds Vbucket 117 2016-04-07T03:43:08.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13620 Milliseconds Vbucket 117 2016-04-07T03:43:08.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13650 Milliseconds Vbucket 117 2016-04-07T03:43:08.396-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5780000 QueuedCount 50000 2016-04-07T03:43:08.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13680 Milliseconds Vbucket 117 2016-04-07T03:43:08.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13710 Milliseconds Vbucket 117 2016-04-07T03:43:08.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13740 Milliseconds Vbucket 117 2016-04-07T03:43:08.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13770 Milliseconds Vbucket 117 2016-04-07T03:43:08.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13800 Milliseconds Vbucket 117 2016-04-07T03:43:08.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13830 Milliseconds Vbucket 117 2016-04-07T03:43:08.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13860 Milliseconds Vbucket 117 2016-04-07T03:43:08.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13890 Milliseconds Vbucket 117 2016-04-07T03:43:08.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13920 Milliseconds Vbucket 117 2016-04-07T03:43:08.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13950 Milliseconds Vbucket 117 2016-04-07T03:43:08.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13980 Milliseconds Vbucket 117 2016-04-07T03:43:08.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14010 Milliseconds Vbucket 117 2016-04-07T03:43:08.74-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5790000 QueuedCount 45062 2016-04-07T03:43:08.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14040 Milliseconds Vbucket 117 2016-04-07T03:43:08.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14070 Milliseconds Vbucket 117 2016-04-07T03:43:08.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14100 Milliseconds Vbucket 117 2016-04-07T03:43:08.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14130 Milliseconds Vbucket 117 2016-04-07T03:43:08.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14160 Milliseconds Vbucket 117 2016-04-07T03:43:08.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14190 Milliseconds Vbucket 117 2016-04-07T03:43:08.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14220 Milliseconds Vbucket 117 2016-04-07T03:43:08.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14250 Milliseconds Vbucket 117 2016-04-07T03:43:08.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14280 Milliseconds Vbucket 117 2016-04-07T03:43:09.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14310 Milliseconds Vbucket 117 2016-04-07T03:43:09.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14340 Milliseconds Vbucket 117 2016-04-07T03:43:09.087-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5800000 QueuedCount 35062 2016-04-07T03:43:09.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14370 Milliseconds Vbucket 117 2016-04-07T03:43:09.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14400 Milliseconds Vbucket 117 2016-04-07T03:43:09.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14430 Milliseconds Vbucket 117 2016-04-07T03:43:09.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14460 Milliseconds Vbucket 117 2016-04-07T03:43:09.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14490 Milliseconds Vbucket 117 2016-04-07T03:43:09.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14520 Milliseconds Vbucket 117 2016-04-07T03:43:09.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14550 Milliseconds Vbucket 117 2016-04-07T03:43:09.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14580 Milliseconds Vbucket 117 2016-04-07T03:43:09.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14610 Milliseconds Vbucket 117 2016-04-07T03:43:09.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14640 Milliseconds Vbucket 117 2016-04-07T03:43:09.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14670 Milliseconds Vbucket 117 2016-04-07T03:43:09.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14700 Milliseconds Vbucket 117 2016-04-07T03:43:09.424-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5810000 QueuedCount 25062 2016-04-07T03:43:09.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14730 Milliseconds Vbucket 117 2016-04-07T03:43:09.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14760 Milliseconds Vbucket 117 2016-04-07T03:43:09.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14790 Milliseconds Vbucket 117 2016-04-07T03:43:09.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14820 Milliseconds Vbucket 117 2016-04-07T03:43:09.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14850 Milliseconds Vbucket 117 2016-04-07T03:43:09.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14880 Milliseconds Vbucket 117 2016-04-07T03:43:09.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14910 Milliseconds Vbucket 117 2016-04-07T03:43:09.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14940 Milliseconds Vbucket 117 2016-04-07T03:43:09.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14970 Milliseconds Vbucket 117 2016-04-07T03:43:09.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15000 Milliseconds Vbucket 117 2016-04-07T03:43:09.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15030 Milliseconds Vbucket 117 2016-04-07T03:43:09.754-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5820000 QueuedCount 15062 2016-04-07T03:43:09.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15060 Milliseconds Vbucket 117 2016-04-07T03:43:09.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15090 Milliseconds Vbucket 117 2016-04-07T03:43:09.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15120 Milliseconds Vbucket 117 2016-04-07T03:43:09.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15150 Milliseconds Vbucket 117 2016-04-07T03:43:09.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15180 Milliseconds Vbucket 117 2016-04-07T03:43:09.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15210 Milliseconds Vbucket 117 2016-04-07T03:43:09.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15240 Milliseconds Vbucket 117 2016-04-07T03:43:09.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15270 Milliseconds Vbucket 117 2016-04-07T03:43:10.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15300 Milliseconds Vbucket 117 2016-04-07T03:43:10.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15330 Milliseconds Vbucket 117 2016-04-07T03:43:10.076-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5830000 QueuedCount 5062 2016-04-07T03:43:10.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15360 Milliseconds Vbucket 117 2016-04-07T03:43:10.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15390 Milliseconds Vbucket 117 2016-04-07T03:43:10.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15420 Milliseconds Vbucket 117 2016-04-07T03:43:10.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15450 Milliseconds Vbucket 117 2016-04-07T03:43:10.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15480 Milliseconds Vbucket 117 2016-04-07T03:43:10.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15510 Milliseconds Vbucket 117 2016-04-07T03:43:10.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15540 Milliseconds Vbucket 117 2016-04-07T03:43:10.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15570 Milliseconds Vbucket 117 2016-04-07T03:43:10.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15600 Milliseconds Vbucket 117 2016-04-07T03:43:10.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15630 Milliseconds Vbucket 117 2016-04-07T03:43:10.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15660 Milliseconds Vbucket 117 2016-04-07T03:43:10.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15690 Milliseconds Vbucket 117 2016-04-07T03:43:10.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15720 Milliseconds Vbucket 117 2016-04-07T03:43:10.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15750 Milliseconds Vbucket 117 2016-04-07T03:43:10.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15780 Milliseconds Vbucket 117 2016-04-07T03:43:10.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15810 Milliseconds Vbucket 117 2016-04-07T03:43:10.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15840 Milliseconds Vbucket 117 2016-04-07T03:43:10.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15870 Milliseconds Vbucket 117 2016-04-07T03:43:10.597-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.166µs CommitTime 360.131778ms TotalFlushTime 3m21.830217509s TotalCommitTime 8.295776443s 2016-04-07T03:43:10.597-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 5835062, 5835062, 37 committed:true 2016-04-07T03:43:10.597-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 5835062, 5835062, 37 committed:true) SnapCreateDur 370.177141ms SnapOpenDur 39.554µs 2016-04-07T03:43:10.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15900 Milliseconds Vbucket 117 2016-04-07T03:43:10.639-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:10.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15930 Milliseconds Vbucket 117 2016-04-07T03:43:10.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15960 Milliseconds Vbucket 117 2016-04-07T03:43:10.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15990 Milliseconds Vbucket 117 2016-04-07T03:43:10.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16020 Milliseconds Vbucket 117 2016-04-07T03:43:10.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16050 Milliseconds Vbucket 117 2016-04-07T03:43:10.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16080 Milliseconds Vbucket 117 2016-04-07T03:43:10.81-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5840000 QueuedCount 50000 2016-04-07T03:43:10.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16110 Milliseconds Vbucket 117 2016-04-07T03:43:10.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16140 Milliseconds Vbucket 117 2016-04-07T03:43:10.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16170 Milliseconds Vbucket 117 2016-04-07T03:43:10.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16200 Milliseconds Vbucket 117 2016-04-07T03:43:10.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16230 Milliseconds Vbucket 117 2016-04-07T03:43:10.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16260 Milliseconds Vbucket 117 2016-04-07T03:43:11.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16290 Milliseconds Vbucket 117 2016-04-07T03:43:11.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16320 Milliseconds Vbucket 117 2016-04-07T03:43:11.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16350 Milliseconds Vbucket 117 2016-04-07T03:43:11.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16380 Milliseconds Vbucket 117 2016-04-07T03:43:11.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16410 Milliseconds Vbucket 117 2016-04-07T03:43:11.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16440 Milliseconds Vbucket 117 2016-04-07T03:43:11.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16470 Milliseconds Vbucket 117 2016-04-07T03:43:11.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16500 Milliseconds Vbucket 117 2016-04-07T03:43:11.223-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5850000 QueuedCount 50000 2016-04-07T03:43:11.238-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:11.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16530 Milliseconds Vbucket 117 2016-04-07T03:43:11.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16560 Milliseconds Vbucket 117 2016-04-07T03:43:11.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16590 Milliseconds Vbucket 117 2016-04-07T03:43:11.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16620 Milliseconds Vbucket 117 2016-04-07T03:43:11.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16650 Milliseconds Vbucket 117 2016-04-07T03:43:11.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16680 Milliseconds Vbucket 117 2016-04-07T03:43:11.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16710 Milliseconds Vbucket 117 2016-04-07T03:43:11.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16740 Milliseconds Vbucket 117 2016-04-07T03:43:11.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16770 Milliseconds Vbucket 117 2016-04-07T03:43:11.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16800 Milliseconds Vbucket 117 2016-04-07T03:43:11.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16830 Milliseconds Vbucket 117 2016-04-07T03:43:11.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16860 Milliseconds Vbucket 117 2016-04-07T03:43:11.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16890 Milliseconds Vbucket 117 2016-04-07T03:43:11.628-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5860000 QueuedCount 50000 2016-04-07T03:43:11.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16920 Milliseconds Vbucket 117 2016-04-07T03:43:11.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16950 Milliseconds Vbucket 117 2016-04-07T03:43:11.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16980 Milliseconds Vbucket 117 2016-04-07T03:43:11.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17010 Milliseconds Vbucket 117 2016-04-07T03:43:11.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17040 Milliseconds Vbucket 117 2016-04-07T03:43:11.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17070 Milliseconds Vbucket 117 2016-04-07T03:43:11.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17100 Milliseconds Vbucket 117 2016-04-07T03:43:11.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17130 Milliseconds Vbucket 117 2016-04-07T03:43:11.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17160 Milliseconds Vbucket 117 2016-04-07T03:43:11.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17190 Milliseconds Vbucket 117 2016-04-07T03:43:11.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17220 Milliseconds Vbucket 117 2016-04-07T03:43:11.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17250 Milliseconds Vbucket 117 2016-04-07T03:43:11.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17280 Milliseconds Vbucket 117 2016-04-07T03:43:12.021-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5870000 QueuedCount 50000 2016-04-07T03:43:12.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17310 Milliseconds Vbucket 117 2016-04-07T03:43:12.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17340 Milliseconds Vbucket 117 2016-04-07T03:43:12.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17370 Milliseconds Vbucket 117 2016-04-07T03:43:12.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17400 Milliseconds Vbucket 117 2016-04-07T03:43:12.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17430 Milliseconds Vbucket 117 2016-04-07T03:43:12.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17460 Milliseconds Vbucket 117 2016-04-07T03:43:12.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17490 Milliseconds Vbucket 117 2016-04-07T03:43:12.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17520 Milliseconds Vbucket 117 2016-04-07T03:43:12.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17550 Milliseconds Vbucket 117 2016-04-07T03:43:12.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17580 Milliseconds Vbucket 117 2016-04-07T03:43:12.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17610 Milliseconds Vbucket 117 2016-04-07T03:43:12.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17640 Milliseconds Vbucket 117 2016-04-07T03:43:12.362-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5880000 QueuedCount 50000 2016-04-07T03:43:12.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17670 Milliseconds Vbucket 117 2016-04-07T03:43:12.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17700 Milliseconds Vbucket 117 2016-04-07T03:43:12.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17730 Milliseconds Vbucket 117 2016-04-07T03:43:12.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17760 Milliseconds Vbucket 117 2016-04-07T03:43:12.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17790 Milliseconds Vbucket 117 2016-04-07T03:43:12.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17820 Milliseconds Vbucket 117 2016-04-07T03:43:12.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17850 Milliseconds Vbucket 117 2016-04-07T03:43:12.595-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:12.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17880 Milliseconds Vbucket 117 2016-04-07T03:43:12.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17910 Milliseconds Vbucket 117 2016-04-07T03:43:12.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17940 Milliseconds Vbucket 117 2016-04-07T03:43:12.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17970 Milliseconds Vbucket 117 2016-04-07T03:43:12.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18000 Milliseconds Vbucket 117 2016-04-07T03:43:12.743-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5890000 QueuedCount 50000 2016-04-07T03:43:12.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18030 Milliseconds Vbucket 117 2016-04-07T03:43:12.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18060 Milliseconds Vbucket 117 2016-04-07T03:43:12.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18090 Milliseconds Vbucket 117 2016-04-07T03:43:12.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18120 Milliseconds Vbucket 117 2016-04-07T03:43:12.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18150 Milliseconds Vbucket 117 2016-04-07T03:43:12.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18180 Milliseconds Vbucket 117 2016-04-07T03:43:12.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18210 Milliseconds Vbucket 117 2016-04-07T03:43:12.94-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8343126,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":5205275427,"bucket-1:myindex:avg_ts_items_count":131128,"bucket-1:myindex:build_progress":29,"bucket-1:myindex:data_size":1865560100,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":6304501760,"bucket-1:myindex:flush_queue_size":50032,"bucket-1:myindex:frag_percent":70,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":617628585,"bucket-1:myindex:items_count":5835062,"bucket-1:myindex:num_commits":37,"bucket-1:myindex:num_compactions":2,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":5901178,"bucket-1:myindex:num_docs_pending":5712403,"bucket-1:myindex:num_docs_queued":8386419,"bucket-1:myindex:num_flush_queued":5944500,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":37,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":5406477789,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"37 8295776443 2164082118470455257","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"5894469 38170631839 610424241360619319","bucket-1:myindex:timings/storage_handle_open":"37 892117 22226556593","bucket-1:myindex:timings/storage_info":"149 1330297 31820774669","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"38 49158470 722889236168920","bucket-1:myindex:timings/storage_meta_set":"37 341663313 107820345906455113","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"11788936 162072048555 5590908112792166675","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":14287595,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":301,"memory_quota":31457280000,"memory_used":2566201344,"needs_restart":false,"num_connections":0,"timings/stats_response":"1488 131214280 19926576286588"} 2016-04-07T03:43:12.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18240 Milliseconds Vbucket 117 2016-04-07T03:43:12.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18270 Milliseconds Vbucket 117 2016-04-07T03:43:13.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18300 Milliseconds Vbucket 117 2016-04-07T03:43:13.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18330 Milliseconds Vbucket 117 2016-04-07T03:43:13.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18360 Milliseconds Vbucket 117 2016-04-07T03:43:13.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18390 Milliseconds Vbucket 117 2016-04-07T03:43:13.123-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5900000 QueuedCount 50000 2016-04-07T03:43:13.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18420 Milliseconds Vbucket 117 2016-04-07T03:43:13.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18450 Milliseconds Vbucket 117 2016-04-07T03:43:13.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18480 Milliseconds Vbucket 117 2016-04-07T03:43:13.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18510 Milliseconds Vbucket 117 2016-04-07T03:43:13.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18540 Milliseconds Vbucket 117 2016-04-07T03:43:13.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18570 Milliseconds Vbucket 117 2016-04-07T03:43:13.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18600 Milliseconds Vbucket 117 2016-04-07T03:43:13.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18630 Milliseconds Vbucket 117 2016-04-07T03:43:13.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18660 Milliseconds Vbucket 117 2016-04-07T03:43:13.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18690 Milliseconds Vbucket 117 2016-04-07T03:43:13.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18720 Milliseconds Vbucket 117 2016-04-07T03:43:13.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18750 Milliseconds Vbucket 117 2016-04-07T03:43:13.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18780 Milliseconds Vbucket 117 2016-04-07T03:43:13.498-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5910000 QueuedCount 50000 2016-04-07T03:43:13.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18810 Milliseconds Vbucket 117 2016-04-07T03:43:13.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18840 Milliseconds Vbucket 117 2016-04-07T03:43:13.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18870 Milliseconds Vbucket 117 2016-04-07T03:43:13.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18900 Milliseconds Vbucket 117 2016-04-07T03:43:13.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18930 Milliseconds Vbucket 117 2016-04-07T03:43:13.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18960 Milliseconds Vbucket 117 2016-04-07T03:43:13.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18990 Milliseconds Vbucket 117 2016-04-07T03:43:13.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19020 Milliseconds Vbucket 117 2016-04-07T03:43:13.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19050 Milliseconds Vbucket 117 2016-04-07T03:43:13.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19080 Milliseconds Vbucket 117 2016-04-07T03:43:13.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19110 Milliseconds Vbucket 117 2016-04-07T03:43:13.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19140 Milliseconds Vbucket 117 2016-04-07T03:43:13.873-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5920000 QueuedCount 50000 2016-04-07T03:43:13.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19170 Milliseconds Vbucket 117 2016-04-07T03:43:13.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19200 Milliseconds Vbucket 117 2016-04-07T03:43:13.945-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:13.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19230 Milliseconds Vbucket 117 2016-04-07T03:43:13.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19260 Milliseconds Vbucket 117 2016-04-07T03:43:14.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19290 Milliseconds Vbucket 117 2016-04-07T03:43:14.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19320 Milliseconds Vbucket 117 2016-04-07T03:43:14.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19350 Milliseconds Vbucket 117 2016-04-07T03:43:14.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19380 Milliseconds Vbucket 117 2016-04-07T03:43:14.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19410 Milliseconds Vbucket 117 2016-04-07T03:43:14.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19440 Milliseconds Vbucket 117 2016-04-07T03:43:14.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19470 Milliseconds Vbucket 117 2016-04-07T03:43:14.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19500 Milliseconds Vbucket 117 2016-04-07T03:43:14.239-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5930000 QueuedCount 50000 2016-04-07T03:43:14.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19530 Milliseconds Vbucket 117 2016-04-07T03:43:14.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19560 Milliseconds Vbucket 117 2016-04-07T03:43:14.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19590 Milliseconds Vbucket 117 2016-04-07T03:43:14.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19620 Milliseconds Vbucket 117 2016-04-07T03:43:14.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19650 Milliseconds Vbucket 117 2016-04-07T03:43:14.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19680 Milliseconds Vbucket 117 2016-04-07T03:43:14.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19710 Milliseconds Vbucket 117 2016-04-07T03:43:14.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19740 Milliseconds Vbucket 117 2016-04-07T03:43:14.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19770 Milliseconds Vbucket 117 2016-04-07T03:43:14.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19800 Milliseconds Vbucket 117 2016-04-07T03:43:14.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19830 Milliseconds Vbucket 117 2016-04-07T03:43:14.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19860 Milliseconds Vbucket 117 2016-04-07T03:43:14.605-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5940000 QueuedCount 50000 2016-04-07T03:43:14.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19890 Milliseconds Vbucket 117 2016-04-07T03:43:14.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19920 Milliseconds Vbucket 117 2016-04-07T03:43:14.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19950 Milliseconds Vbucket 117 2016-04-07T03:43:14.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19980 Milliseconds Vbucket 117 2016-04-07T03:43:14.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20010 Milliseconds Vbucket 117 2016-04-07T03:43:14.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20040 Milliseconds Vbucket 117 2016-04-07T03:43:14.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20070 Milliseconds Vbucket 117 2016-04-07T03:43:14.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20100 Milliseconds Vbucket 117 2016-04-07T03:43:14.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20130 Milliseconds Vbucket 117 2016-04-07T03:43:14.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20160 Milliseconds Vbucket 117 2016-04-07T03:43:14.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20190 Milliseconds Vbucket 117 2016-04-07T03:43:14.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20220 Milliseconds Vbucket 117 2016-04-07T03:43:14.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20250 Milliseconds Vbucket 117 2016-04-07T03:43:14.98-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5950000 QueuedCount 50000 2016-04-07T03:43:14.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20280 Milliseconds Vbucket 117 2016-04-07T03:43:15.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20310 Milliseconds Vbucket 117 2016-04-07T03:43:15.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20340 Milliseconds Vbucket 117 2016-04-07T03:43:15.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20370 Milliseconds Vbucket 117 2016-04-07T03:43:15.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20400 Milliseconds Vbucket 117 2016-04-07T03:43:15.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20430 Milliseconds Vbucket 117 2016-04-07T03:43:15.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20460 Milliseconds Vbucket 117 2016-04-07T03:43:15.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20490 Milliseconds Vbucket 117 2016-04-07T03:43:15.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20520 Milliseconds Vbucket 117 2016-04-07T03:43:15.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20550 Milliseconds Vbucket 117 2016-04-07T03:43:15.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20580 Milliseconds Vbucket 117 2016-04-07T03:43:15.306-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5960000 QueuedCount 41728 2016-04-07T03:43:15.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20610 Milliseconds Vbucket 117 2016-04-07T03:43:15.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20640 Milliseconds Vbucket 117 2016-04-07T03:43:15.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20670 Milliseconds Vbucket 117 2016-04-07T03:43:15.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20700 Milliseconds Vbucket 117 2016-04-07T03:43:15.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20730 Milliseconds Vbucket 117 2016-04-07T03:43:15.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20760 Milliseconds Vbucket 117 2016-04-07T03:43:15.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20790 Milliseconds Vbucket 117 2016-04-07T03:43:15.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20820 Milliseconds Vbucket 117 2016-04-07T03:43:15.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20850 Milliseconds Vbucket 117 2016-04-07T03:43:15.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20880 Milliseconds Vbucket 117 2016-04-07T03:43:15.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20910 Milliseconds Vbucket 117 2016-04-07T03:43:15.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20940 Milliseconds Vbucket 117 2016-04-07T03:43:15.665-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5970000 QueuedCount 31728 2016-04-07T03:43:15.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20970 Milliseconds Vbucket 117 2016-04-07T03:43:15.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21000 Milliseconds Vbucket 117 2016-04-07T03:43:15.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21030 Milliseconds Vbucket 117 2016-04-07T03:43:15.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21060 Milliseconds Vbucket 117 2016-04-07T03:43:15.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21090 Milliseconds Vbucket 117 2016-04-07T03:43:15.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21120 Milliseconds Vbucket 117 2016-04-07T03:43:15.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21150 Milliseconds Vbucket 117 2016-04-07T03:43:15.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21180 Milliseconds Vbucket 117 2016-04-07T03:43:15.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21210 Milliseconds Vbucket 117 2016-04-07T03:43:15.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21240 Milliseconds Vbucket 117 2016-04-07T03:43:15.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21270 Milliseconds Vbucket 117 2016-04-07T03:43:16.014-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5980000 QueuedCount 21728 2016-04-07T03:43:16.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21300 Milliseconds Vbucket 117 2016-04-07T03:43:16.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21330 Milliseconds Vbucket 117 2016-04-07T03:43:16.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21360 Milliseconds Vbucket 117 2016-04-07T03:43:16.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21390 Milliseconds Vbucket 117 2016-04-07T03:43:16.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21420 Milliseconds Vbucket 117 2016-04-07T03:43:16.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21450 Milliseconds Vbucket 117 2016-04-07T03:43:16.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21480 Milliseconds Vbucket 117 2016-04-07T03:43:16.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21510 Milliseconds Vbucket 117 2016-04-07T03:43:16.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21540 Milliseconds Vbucket 117 2016-04-07T03:43:16.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21570 Milliseconds Vbucket 117 2016-04-07T03:43:16.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21600 Milliseconds Vbucket 117 2016-04-07T03:43:16.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21630 Milliseconds Vbucket 117 2016-04-07T03:43:16.348-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 5990000 QueuedCount 11728 2016-04-07T03:43:16.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21660 Milliseconds Vbucket 117 2016-04-07T03:43:16.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21690 Milliseconds Vbucket 117 2016-04-07T03:43:16.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21720 Milliseconds Vbucket 117 2016-04-07T03:43:16.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21750 Milliseconds Vbucket 117 2016-04-07T03:43:16.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21780 Milliseconds Vbucket 117 2016-04-07T03:43:16.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21810 Milliseconds Vbucket 117 2016-04-07T03:43:16.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21840 Milliseconds Vbucket 117 2016-04-07T03:43:16.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21870 Milliseconds Vbucket 117 2016-04-07T03:43:16.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21900 Milliseconds Vbucket 117 2016-04-07T03:43:16.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21930 Milliseconds Vbucket 117 2016-04-07T03:43:16.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21960 Milliseconds Vbucket 117 2016-04-07T03:43:16.678-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6000000 QueuedCount 1728 2016-04-07T03:43:16.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21990 Milliseconds Vbucket 117 2016-04-07T03:43:16.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22020 Milliseconds Vbucket 117 2016-04-07T03:43:16.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22050 Milliseconds Vbucket 117 2016-04-07T03:43:16.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22080 Milliseconds Vbucket 117 2016-04-07T03:43:16.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22110 Milliseconds Vbucket 117 2016-04-07T03:43:16.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22140 Milliseconds Vbucket 117 2016-04-07T03:43:16.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22170 Milliseconds Vbucket 117 2016-04-07T03:43:16.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22200 Milliseconds Vbucket 117 2016-04-07T03:43:16.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22230 Milliseconds Vbucket 117 2016-04-07T03:43:16.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22260 Milliseconds Vbucket 117 2016-04-07T03:43:17.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22290 Milliseconds Vbucket 117 2016-04-07T03:43:17.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22320 Milliseconds Vbucket 117 2016-04-07T03:43:17.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22350 Milliseconds Vbucket 117 2016-04-07T03:43:17.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22380 Milliseconds Vbucket 117 2016-04-07T03:43:17.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22410 Milliseconds Vbucket 117 2016-04-07T03:43:17.135-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.131µs CommitTime 379.972837ms TotalFlushTime 3m27.676647564s TotalCommitTime 8.67574928s 2016-04-07T03:43:17.135-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 6001728, 6001728, 38 committed:true 2016-04-07T03:43:17.135-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 6001728, 6001728, 38 committed:true) SnapCreateDur 390.373531ms SnapOpenDur 52.269µs 2016-04-07T03:43:17.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22440 Milliseconds Vbucket 117 2016-04-07T03:43:17.178-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:17.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22470 Milliseconds Vbucket 117 2016-04-07T03:43:17.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22500 Milliseconds Vbucket 117 2016-04-07T03:43:17.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22530 Milliseconds Vbucket 117 2016-04-07T03:43:17.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22560 Milliseconds Vbucket 117 2016-04-07T03:43:17.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22590 Milliseconds Vbucket 117 2016-04-07T03:43:17.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22620 Milliseconds Vbucket 117 2016-04-07T03:43:17.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22650 Milliseconds Vbucket 117 2016-04-07T03:43:17.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22680 Milliseconds Vbucket 117 2016-04-07T03:43:17.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22710 Milliseconds Vbucket 117 2016-04-07T03:43:17.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22740 Milliseconds Vbucket 117 2016-04-07T03:43:17.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22770 Milliseconds Vbucket 117 2016-04-07T03:43:17.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22800 Milliseconds Vbucket 117 2016-04-07T03:43:17.523-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6010000 QueuedCount 50000 2016-04-07T03:43:17.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22830 Milliseconds Vbucket 117 2016-04-07T03:43:17.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22860 Milliseconds Vbucket 117 2016-04-07T03:43:17.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22890 Milliseconds Vbucket 117 2016-04-07T03:43:17.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22920 Milliseconds Vbucket 117 2016-04-07T03:43:17.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22950 Milliseconds Vbucket 117 2016-04-07T03:43:17.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22980 Milliseconds Vbucket 117 2016-04-07T03:43:17.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23010 Milliseconds Vbucket 117 2016-04-07T03:43:17.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23040 Milliseconds Vbucket 117 2016-04-07T03:43:17.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23070 Milliseconds Vbucket 117 2016-04-07T03:43:17.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23100 Milliseconds Vbucket 117 2016-04-07T03:43:17.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23130 Milliseconds Vbucket 117 2016-04-07T03:43:17.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23160 Milliseconds Vbucket 117 2016-04-07T03:43:17.897-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6020000 QueuedCount 50000 2016-04-07T03:43:17.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23190 Milliseconds Vbucket 117 2016-04-07T03:43:17.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23220 Milliseconds Vbucket 117 2016-04-07T03:43:17.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23250 Milliseconds Vbucket 117 2016-04-07T03:43:17.968-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:17.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23280 Milliseconds Vbucket 117 2016-04-07T03:43:18.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23310 Milliseconds Vbucket 117 2016-04-07T03:43:18.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23340 Milliseconds Vbucket 117 2016-04-07T03:43:18.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23370 Milliseconds Vbucket 117 2016-04-07T03:43:18.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23400 Milliseconds Vbucket 117 2016-04-07T03:43:18.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23430 Milliseconds Vbucket 117 2016-04-07T03:43:18.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23460 Milliseconds Vbucket 117 2016-04-07T03:43:18.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23490 Milliseconds Vbucket 117 2016-04-07T03:43:18.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23520 Milliseconds Vbucket 117 2016-04-07T03:43:18.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23550 Milliseconds Vbucket 117 2016-04-07T03:43:18.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23580 Milliseconds Vbucket 117 2016-04-07T03:43:18.301-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6030000 QueuedCount 50000 2016-04-07T03:43:18.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23610 Milliseconds Vbucket 117 2016-04-07T03:43:18.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23640 Milliseconds Vbucket 117 2016-04-07T03:43:18.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23670 Milliseconds Vbucket 117 2016-04-07T03:43:18.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23700 Milliseconds Vbucket 117 2016-04-07T03:43:18.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23730 Milliseconds Vbucket 117 2016-04-07T03:43:18.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23760 Milliseconds Vbucket 117 2016-04-07T03:43:18.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23790 Milliseconds Vbucket 117 2016-04-07T03:43:18.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23820 Milliseconds Vbucket 117 2016-04-07T03:43:18.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23850 Milliseconds Vbucket 117 2016-04-07T03:43:18.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23880 Milliseconds Vbucket 117 2016-04-07T03:43:18.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23910 Milliseconds Vbucket 117 2016-04-07T03:43:18.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23940 Milliseconds Vbucket 117 2016-04-07T03:43:18.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23970 Milliseconds Vbucket 117 2016-04-07T03:43:18.69-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6040000 QueuedCount 50000 2016-04-07T03:43:18.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24000 Milliseconds Vbucket 117 2016-04-07T03:43:18.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24030 Milliseconds Vbucket 117 2016-04-07T03:43:18.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24060 Milliseconds Vbucket 117 2016-04-07T03:43:18.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24090 Milliseconds Vbucket 117 2016-04-07T03:43:18.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24120 Milliseconds Vbucket 117 2016-04-07T03:43:18.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24150 Milliseconds Vbucket 117 2016-04-07T03:43:18.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24180 Milliseconds Vbucket 117 2016-04-07T03:43:18.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24210 Milliseconds Vbucket 117 2016-04-07T03:43:18.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24240 Milliseconds Vbucket 117 2016-04-07T03:43:18.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24270 Milliseconds Vbucket 117 2016-04-07T03:43:19.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24300 Milliseconds Vbucket 117 2016-04-07T03:43:19.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24330 Milliseconds Vbucket 117 2016-04-07T03:43:19.068-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6050000 QueuedCount 50000 2016-04-07T03:43:19.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24360 Milliseconds Vbucket 117 2016-04-07T03:43:19.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24390 Milliseconds Vbucket 117 2016-04-07T03:43:19.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24420 Milliseconds Vbucket 117 2016-04-07T03:43:19.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24450 Milliseconds Vbucket 117 2016-04-07T03:43:19.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24480 Milliseconds Vbucket 117 2016-04-07T03:43:19.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24510 Milliseconds Vbucket 117 2016-04-07T03:43:19.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24540 Milliseconds Vbucket 117 2016-04-07T03:43:19.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24570 Milliseconds Vbucket 117 2016-04-07T03:43:19.306-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:19.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24600 Milliseconds Vbucket 117 2016-04-07T03:43:19.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24630 Milliseconds Vbucket 117 2016-04-07T03:43:19.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24660 Milliseconds Vbucket 117 2016-04-07T03:43:19.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24690 Milliseconds Vbucket 117 2016-04-07T03:43:19.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24720 Milliseconds Vbucket 117 2016-04-07T03:43:19.458-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6060000 QueuedCount 50000 2016-04-07T03:43:19.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24750 Milliseconds Vbucket 117 2016-04-07T03:43:19.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24780 Milliseconds Vbucket 117 2016-04-07T03:43:19.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24810 Milliseconds Vbucket 117 2016-04-07T03:43:19.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24840 Milliseconds Vbucket 117 2016-04-07T03:43:19.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24870 Milliseconds Vbucket 117 2016-04-07T03:43:19.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24900 Milliseconds Vbucket 117 2016-04-07T03:43:19.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24930 Milliseconds Vbucket 117 2016-04-07T03:43:19.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24960 Milliseconds Vbucket 117 2016-04-07T03:43:19.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24990 Milliseconds Vbucket 117 2016-04-07T03:43:19.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25020 Milliseconds Vbucket 117 2016-04-07T03:43:19.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25050 Milliseconds Vbucket 117 2016-04-07T03:43:19.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25080 Milliseconds Vbucket 117 2016-04-07T03:43:19.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25110 Milliseconds Vbucket 117 2016-04-07T03:43:19.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25140 Milliseconds Vbucket 117 2016-04-07T03:43:19.866-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6070000 QueuedCount 50000 2016-04-07T03:43:19.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25170 Milliseconds Vbucket 117 2016-04-07T03:43:19.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25200 Milliseconds Vbucket 117 2016-04-07T03:43:19.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25230 Milliseconds Vbucket 117 2016-04-07T03:43:19.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25260 Milliseconds Vbucket 117 2016-04-07T03:43:20.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25290 Milliseconds Vbucket 117 2016-04-07T03:43:20.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25320 Milliseconds Vbucket 117 2016-04-07T03:43:20.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25350 Milliseconds Vbucket 117 2016-04-07T03:43:20.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25380 Milliseconds Vbucket 117 2016-04-07T03:43:20.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25410 Milliseconds Vbucket 117 2016-04-07T03:43:20.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25440 Milliseconds Vbucket 117 2016-04-07T03:43:20.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25470 Milliseconds Vbucket 117 2016-04-07T03:43:20.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25500 Milliseconds Vbucket 117 2016-04-07T03:43:20.238-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6080000 QueuedCount 50000 2016-04-07T03:43:20.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25530 Milliseconds Vbucket 117 2016-04-07T03:43:20.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25560 Milliseconds Vbucket 117 2016-04-07T03:43:20.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25590 Milliseconds Vbucket 117 2016-04-07T03:43:20.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25620 Milliseconds Vbucket 117 2016-04-07T03:43:20.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25650 Milliseconds Vbucket 117 2016-04-07T03:43:20.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25680 Milliseconds Vbucket 117 2016-04-07T03:43:20.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25710 Milliseconds Vbucket 117 2016-04-07T03:43:20.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25740 Milliseconds Vbucket 117 2016-04-07T03:43:20.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25770 Milliseconds Vbucket 117 2016-04-07T03:43:20.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25800 Milliseconds Vbucket 117 2016-04-07T03:43:20.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25830 Milliseconds Vbucket 117 2016-04-07T03:43:20.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25860 Milliseconds Vbucket 117 2016-04-07T03:43:20.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25890 Milliseconds Vbucket 117 2016-04-07T03:43:20.613-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6090000 QueuedCount 50000 2016-04-07T03:43:20.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25920 Milliseconds Vbucket 117 2016-04-07T03:43:20.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25950 Milliseconds Vbucket 117 2016-04-07T03:43:20.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25980 Milliseconds Vbucket 117 2016-04-07T03:43:20.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26010 Milliseconds Vbucket 117 2016-04-07T03:43:20.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26040 Milliseconds Vbucket 117 2016-04-07T03:43:20.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26070 Milliseconds Vbucket 117 2016-04-07T03:43:20.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26100 Milliseconds Vbucket 117 2016-04-07T03:43:20.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26130 Milliseconds Vbucket 117 2016-04-07T03:43:20.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26160 Milliseconds Vbucket 117 2016-04-07T03:43:20.891-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6100000 QueuedCount 40964 2016-04-07T03:43:20.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26190 Milliseconds Vbucket 117 2016-04-07T03:43:20.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26220 Milliseconds Vbucket 117 2016-04-07T03:43:20.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26250 Milliseconds Vbucket 117 2016-04-07T03:43:20.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26280 Milliseconds Vbucket 117 2016-04-07T03:43:21.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26310 Milliseconds Vbucket 117 2016-04-07T03:43:21.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26340 Milliseconds Vbucket 117 2016-04-07T03:43:21.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26370 Milliseconds Vbucket 117 2016-04-07T03:43:21.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26400 Milliseconds Vbucket 117 2016-04-07T03:43:21.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26430 Milliseconds Vbucket 117 2016-04-07T03:43:21.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26460 Milliseconds Vbucket 117 2016-04-07T03:43:21.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26490 Milliseconds Vbucket 117 2016-04-07T03:43:21.231-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6110000 QueuedCount 30964 2016-04-07T03:43:21.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26520 Milliseconds Vbucket 117 2016-04-07T03:43:21.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26550 Milliseconds Vbucket 117 2016-04-07T03:43:21.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26580 Milliseconds Vbucket 117 2016-04-07T03:43:21.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26610 Milliseconds Vbucket 117 2016-04-07T03:43:21.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26640 Milliseconds Vbucket 117 2016-04-07T03:43:21.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26670 Milliseconds Vbucket 117 2016-04-07T03:43:21.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26700 Milliseconds Vbucket 117 2016-04-07T03:43:21.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26730 Milliseconds Vbucket 117 2016-04-07T03:43:21.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26760 Milliseconds Vbucket 117 2016-04-07T03:43:21.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26790 Milliseconds Vbucket 117 2016-04-07T03:43:21.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26820 Milliseconds Vbucket 117 2016-04-07T03:43:21.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26850 Milliseconds Vbucket 117 2016-04-07T03:43:21.572-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6120000 QueuedCount 20964 2016-04-07T03:43:21.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26880 Milliseconds Vbucket 117 2016-04-07T03:43:21.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26910 Milliseconds Vbucket 117 2016-04-07T03:43:21.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26940 Milliseconds Vbucket 117 2016-04-07T03:43:21.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26970 Milliseconds Vbucket 117 2016-04-07T03:43:21.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27000 Milliseconds Vbucket 117 2016-04-07T03:43:21.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27030 Milliseconds Vbucket 117 2016-04-07T03:43:21.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27060 Milliseconds Vbucket 117 2016-04-07T03:43:21.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27090 Milliseconds Vbucket 117 2016-04-07T03:43:21.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27120 Milliseconds Vbucket 117 2016-04-07T03:43:21.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27150 Milliseconds Vbucket 117 2016-04-07T03:43:21.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27180 Milliseconds Vbucket 117 2016-04-07T03:43:21.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27210 Milliseconds Vbucket 117 2016-04-07T03:43:21.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27240 Milliseconds Vbucket 117 2016-04-07T03:43:21.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27270 Milliseconds Vbucket 117 2016-04-07T03:43:22.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27300 Milliseconds Vbucket 117 2016-04-07T03:43:22.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27330 Milliseconds Vbucket 117 2016-04-07T03:43:22.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27360 Milliseconds Vbucket 117 2016-04-07T03:43:22.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27390 Milliseconds Vbucket 117 2016-04-07T03:43:22.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27420 Milliseconds Vbucket 117 2016-04-07T03:43:22.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27450 Milliseconds Vbucket 117 2016-04-07T03:43:22.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27480 Milliseconds Vbucket 117 2016-04-07T03:43:22.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27510 Milliseconds Vbucket 117 2016-04-07T03:43:22.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27540 Milliseconds Vbucket 117 2016-04-07T03:43:22.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27570 Milliseconds Vbucket 117 2016-04-07T03:43:22.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27600 Milliseconds Vbucket 117 2016-04-07T03:43:22.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27630 Milliseconds Vbucket 117 2016-04-07T03:43:22.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27660 Milliseconds Vbucket 117 2016-04-07T03:43:22.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27690 Milliseconds Vbucket 117 2016-04-07T03:43:22.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27720 Milliseconds Vbucket 117 2016-04-07T03:43:22.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27750 Milliseconds Vbucket 117 2016-04-07T03:43:22.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27780 Milliseconds Vbucket 117 2016-04-07T03:43:22.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27810 Milliseconds Vbucket 117 2016-04-07T03:43:22.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27840 Milliseconds Vbucket 117 2016-04-07T03:43:22.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27870 Milliseconds Vbucket 117 2016-04-07T03:43:22.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27900 Milliseconds Vbucket 117 2016-04-07T03:43:22.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27930 Milliseconds Vbucket 117 2016-04-07T03:43:22.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27960 Milliseconds Vbucket 117 2016-04-07T03:43:22.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27990 Milliseconds Vbucket 117 2016-04-07T03:43:22.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28020 Milliseconds Vbucket 117 2016-04-07T03:43:22.752-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6130000 QueuedCount 10964 2016-04-07T03:43:22.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28050 Milliseconds Vbucket 117 2016-04-07T03:43:22.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28080 Milliseconds Vbucket 117 2016-04-07T03:43:22.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28110 Milliseconds Vbucket 117 2016-04-07T03:43:22.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28140 Milliseconds Vbucket 117 2016-04-07T03:43:22.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28170 Milliseconds Vbucket 117 2016-04-07T03:43:22.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28200 Milliseconds Vbucket 117 2016-04-07T03:43:22.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28230 Milliseconds Vbucket 117 2016-04-07T03:43:22.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28260 Milliseconds Vbucket 117 2016-04-07T03:43:23.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28290 Milliseconds Vbucket 117 2016-04-07T03:43:23.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28320 Milliseconds Vbucket 117 2016-04-07T03:43:23.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28350 Milliseconds Vbucket 117 2016-04-07T03:43:23.083-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6140000 QueuedCount 964 2016-04-07T03:43:23.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28380 Milliseconds Vbucket 117 2016-04-07T03:43:23.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28410 Milliseconds Vbucket 117 2016-04-07T03:43:23.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28440 Milliseconds Vbucket 117 2016-04-07T03:43:23.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28470 Milliseconds Vbucket 117 2016-04-07T03:43:23.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28500 Milliseconds Vbucket 117 2016-04-07T03:43:23.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28530 Milliseconds Vbucket 117 2016-04-07T03:43:23.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28560 Milliseconds Vbucket 117 2016-04-07T03:43:23.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28590 Milliseconds Vbucket 117 2016-04-07T03:43:23.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28620 Milliseconds Vbucket 117 2016-04-07T03:43:23.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28650 Milliseconds Vbucket 117 2016-04-07T03:43:23.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28680 Milliseconds Vbucket 117 2016-04-07T03:43:23.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28710 Milliseconds Vbucket 117 2016-04-07T03:43:23.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28740 Milliseconds Vbucket 117 2016-04-07T03:43:23.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28770 Milliseconds Vbucket 117 2016-04-07T03:43:23.488-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.632µs CommitTime 382.028943ms TotalFlushTime 3m33.391250984s TotalCommitTime 9.057778223s 2016-04-07T03:43:23.488-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 6140964, 6140964, 39 committed:true 2016-04-07T03:43:23.488-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 6140964, 6140964, 39 committed:true) SnapCreateDur 392.822772ms SnapOpenDur 54.367µs 2016-04-07T03:43:23.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28800 Milliseconds Vbucket 117 2016-04-07T03:43:23.529-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:23.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28830 Milliseconds Vbucket 117 2016-04-07T03:43:23.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28860 Milliseconds Vbucket 117 2016-04-07T03:43:23.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28890 Milliseconds Vbucket 117 2016-04-07T03:43:23.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28920 Milliseconds Vbucket 117 2016-04-07T03:43:23.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28950 Milliseconds Vbucket 117 2016-04-07T03:43:23.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28980 Milliseconds Vbucket 117 2016-04-07T03:43:23.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29010 Milliseconds Vbucket 117 2016-04-07T03:43:23.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29040 Milliseconds Vbucket 117 2016-04-07T03:43:23.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29070 Milliseconds Vbucket 117 2016-04-07T03:43:23.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29100 Milliseconds Vbucket 117 2016-04-07T03:43:23.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29130 Milliseconds Vbucket 117 2016-04-07T03:43:23.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29160 Milliseconds Vbucket 117 2016-04-07T03:43:23.882-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6150000 QueuedCount 50000 2016-04-07T03:43:23.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29190 Milliseconds Vbucket 117 2016-04-07T03:43:23.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29220 Milliseconds Vbucket 117 2016-04-07T03:43:23.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29250 Milliseconds Vbucket 117 2016-04-07T03:43:23.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29280 Milliseconds Vbucket 117 2016-04-07T03:43:24.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29310 Milliseconds Vbucket 117 2016-04-07T03:43:24.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29340 Milliseconds Vbucket 117 2016-04-07T03:43:24.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29370 Milliseconds Vbucket 117 2016-04-07T03:43:24.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29400 Milliseconds Vbucket 117 2016-04-07T03:43:24.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29430 Milliseconds Vbucket 117 2016-04-07T03:43:24.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29460 Milliseconds Vbucket 117 2016-04-07T03:43:24.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29490 Milliseconds Vbucket 117 2016-04-07T03:43:24.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29520 Milliseconds Vbucket 117 2016-04-07T03:43:24.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29550 Milliseconds Vbucket 117 2016-04-07T03:43:24.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29580 Milliseconds Vbucket 117 2016-04-07T03:43:24.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29610 Milliseconds Vbucket 117 2016-04-07T03:43:24.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29640 Milliseconds Vbucket 117 2016-04-07T03:43:24.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29670 Milliseconds Vbucket 117 2016-04-07T03:43:24.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29700 Milliseconds Vbucket 117 2016-04-07T03:43:24.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29730 Milliseconds Vbucket 117 2016-04-07T03:43:24.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29760 Milliseconds Vbucket 117 2016-04-07T03:43:24.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29790 Milliseconds Vbucket 117 2016-04-07T03:43:24.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29820 Milliseconds Vbucket 117 2016-04-07T03:43:24.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29850 Milliseconds Vbucket 117 2016-04-07T03:43:24.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29880 Milliseconds Vbucket 117 2016-04-07T03:43:24.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29910 Milliseconds Vbucket 117 2016-04-07T03:43:24.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29940 Milliseconds Vbucket 117 2016-04-07T03:43:24.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29970 Milliseconds Vbucket 117 2016-04-07T03:43:24.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30000 Milliseconds Vbucket 117 2016-04-07T03:43:24.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30030 Milliseconds Vbucket 117 2016-04-07T03:43:24.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30060 Milliseconds Vbucket 117 2016-04-07T03:43:24.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30090 Milliseconds Vbucket 117 2016-04-07T03:43:24.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30120 Milliseconds Vbucket 117 2016-04-07T03:43:24.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30150 Milliseconds Vbucket 117 2016-04-07T03:43:24.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30180 Milliseconds Vbucket 117 2016-04-07T03:43:24.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30210 Milliseconds Vbucket 117 2016-04-07T03:43:24.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30240 Milliseconds Vbucket 117 2016-04-07T03:43:24.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30270 Milliseconds Vbucket 117 2016-04-07T03:43:25.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30300 Milliseconds Vbucket 117 2016-04-07T03:43:25.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30330 Milliseconds Vbucket 117 2016-04-07T03:43:25.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30360 Milliseconds Vbucket 117 2016-04-07T03:43:25.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30390 Milliseconds Vbucket 117 2016-04-07T03:43:25.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30420 Milliseconds Vbucket 117 2016-04-07T03:43:25.156-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:25.16-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6160000 QueuedCount 49647 2016-04-07T03:43:25.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30450 Milliseconds Vbucket 117 2016-04-07T03:43:25.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30480 Milliseconds Vbucket 117 2016-04-07T03:43:25.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30510 Milliseconds Vbucket 117 2016-04-07T03:43:25.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30540 Milliseconds Vbucket 117 2016-04-07T03:43:25.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30570 Milliseconds Vbucket 117 2016-04-07T03:43:25.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30600 Milliseconds Vbucket 117 2016-04-07T03:43:25.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30630 Milliseconds Vbucket 117 2016-04-07T03:43:25.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30660 Milliseconds Vbucket 117 2016-04-07T03:43:25.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30690 Milliseconds Vbucket 117 2016-04-07T03:43:25.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30720 Milliseconds Vbucket 117 2016-04-07T03:43:25.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30750 Milliseconds Vbucket 117 2016-04-07T03:43:25.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30780 Milliseconds Vbucket 117 2016-04-07T03:43:25.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30810 Milliseconds Vbucket 117 2016-04-07T03:43:25.544-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6170000 QueuedCount 50000 2016-04-07T03:43:25.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30840 Milliseconds Vbucket 117 2016-04-07T03:43:25.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30870 Milliseconds Vbucket 117 2016-04-07T03:43:25.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30900 Milliseconds Vbucket 117 2016-04-07T03:43:25.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30930 Milliseconds Vbucket 117 2016-04-07T03:43:25.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30960 Milliseconds Vbucket 117 2016-04-07T03:43:25.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30990 Milliseconds Vbucket 117 2016-04-07T03:43:25.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31020 Milliseconds Vbucket 117 2016-04-07T03:43:25.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31050 Milliseconds Vbucket 117 2016-04-07T03:43:25.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31080 Milliseconds Vbucket 117 2016-04-07T03:43:25.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31110 Milliseconds Vbucket 117 2016-04-07T03:43:25.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31140 Milliseconds Vbucket 117 2016-04-07T03:43:25.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31170 Milliseconds Vbucket 117 2016-04-07T03:43:25.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31200 Milliseconds Vbucket 117 2016-04-07T03:43:25.941-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6180000 QueuedCount 50000 2016-04-07T03:43:25.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31230 Milliseconds Vbucket 117 2016-04-07T03:43:25.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31260 Milliseconds Vbucket 117 2016-04-07T03:43:26.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31290 Milliseconds Vbucket 117 2016-04-07T03:43:26.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31320 Milliseconds Vbucket 117 2016-04-07T03:43:26.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31350 Milliseconds Vbucket 117 2016-04-07T03:43:26.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31380 Milliseconds Vbucket 117 2016-04-07T03:43:26.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31410 Milliseconds Vbucket 117 2016-04-07T03:43:26.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31440 Milliseconds Vbucket 117 2016-04-07T03:43:26.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31470 Milliseconds Vbucket 117 2016-04-07T03:43:26.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31500 Milliseconds Vbucket 117 2016-04-07T03:43:26.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31530 Milliseconds Vbucket 117 2016-04-07T03:43:26.276-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6190000 QueuedCount 50000 2016-04-07T03:43:26.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31560 Milliseconds Vbucket 117 2016-04-07T03:43:26.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31590 Milliseconds Vbucket 117 2016-04-07T03:43:26.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31620 Milliseconds Vbucket 117 2016-04-07T03:43:26.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31650 Milliseconds Vbucket 117 2016-04-07T03:43:26.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31680 Milliseconds Vbucket 117 2016-04-07T03:43:26.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31710 Milliseconds Vbucket 117 2016-04-07T03:43:26.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31740 Milliseconds Vbucket 117 2016-04-07T03:43:26.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31770 Milliseconds Vbucket 117 2016-04-07T03:43:26.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31800 Milliseconds Vbucket 117 2016-04-07T03:43:26.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31830 Milliseconds Vbucket 117 2016-04-07T03:43:26.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31860 Milliseconds Vbucket 117 2016-04-07T03:43:26.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31890 Milliseconds Vbucket 117 2016-04-07T03:43:26.619-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6200000 QueuedCount 44395 2016-04-07T03:43:26.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31920 Milliseconds Vbucket 117 2016-04-07T03:43:26.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31950 Milliseconds Vbucket 117 2016-04-07T03:43:26.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31980 Milliseconds Vbucket 117 2016-04-07T03:43:26.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32010 Milliseconds Vbucket 117 2016-04-07T03:43:26.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32040 Milliseconds Vbucket 117 2016-04-07T03:43:26.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32070 Milliseconds Vbucket 117 2016-04-07T03:43:26.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32100 Milliseconds Vbucket 117 2016-04-07T03:43:26.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32130 Milliseconds Vbucket 117 2016-04-07T03:43:26.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32160 Milliseconds Vbucket 117 2016-04-07T03:43:26.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32190 Milliseconds Vbucket 117 2016-04-07T03:43:26.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32220 Milliseconds Vbucket 117 2016-04-07T03:43:26.945-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6210000 QueuedCount 34395 2016-04-07T03:43:26.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32250 Milliseconds Vbucket 117 2016-04-07T03:43:26.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32280 Milliseconds Vbucket 117 2016-04-07T03:43:27.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32310 Milliseconds Vbucket 117 2016-04-07T03:43:27.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32340 Milliseconds Vbucket 117 2016-04-07T03:43:27.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32370 Milliseconds Vbucket 117 2016-04-07T03:43:27.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32400 Milliseconds Vbucket 117 2016-04-07T03:43:27.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32430 Milliseconds Vbucket 117 2016-04-07T03:43:27.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32460 Milliseconds Vbucket 117 2016-04-07T03:43:27.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32490 Milliseconds Vbucket 117 2016-04-07T03:43:27.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32520 Milliseconds Vbucket 117 2016-04-07T03:43:27.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32550 Milliseconds Vbucket 117 2016-04-07T03:43:27.282-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6220000 QueuedCount 24395 2016-04-07T03:43:27.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32580 Milliseconds Vbucket 117 2016-04-07T03:43:27.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32610 Milliseconds Vbucket 117 2016-04-07T03:43:27.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32640 Milliseconds Vbucket 117 2016-04-07T03:43:27.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32670 Milliseconds Vbucket 117 2016-04-07T03:43:27.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32700 Milliseconds Vbucket 117 2016-04-07T03:43:27.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32730 Milliseconds Vbucket 117 2016-04-07T03:43:27.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32760 Milliseconds Vbucket 117 2016-04-07T03:43:27.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32790 Milliseconds Vbucket 117 2016-04-07T03:43:27.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32820 Milliseconds Vbucket 117 2016-04-07T03:43:27.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32850 Milliseconds Vbucket 117 2016-04-07T03:43:27.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32880 Milliseconds Vbucket 117 2016-04-07T03:43:27.624-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6230000 QueuedCount 14395 2016-04-07T03:43:27.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32910 Milliseconds Vbucket 117 2016-04-07T03:43:27.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32940 Milliseconds Vbucket 117 2016-04-07T03:43:27.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32970 Milliseconds Vbucket 117 2016-04-07T03:43:27.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33000 Milliseconds Vbucket 117 2016-04-07T03:43:27.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33030 Milliseconds Vbucket 117 2016-04-07T03:43:27.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33060 Milliseconds Vbucket 117 2016-04-07T03:43:27.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33090 Milliseconds Vbucket 117 2016-04-07T03:43:27.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33120 Milliseconds Vbucket 117 2016-04-07T03:43:27.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33150 Milliseconds Vbucket 117 2016-04-07T03:43:27.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33180 Milliseconds Vbucket 117 2016-04-07T03:43:27.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33210 Milliseconds Vbucket 117 2016-04-07T03:43:27.94-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6240000 QueuedCount 4395 2016-04-07T03:43:27.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33240 Milliseconds Vbucket 117 2016-04-07T03:43:27.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33270 Milliseconds Vbucket 117 2016-04-07T03:43:28.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33300 Milliseconds Vbucket 117 2016-04-07T03:43:28.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33330 Milliseconds Vbucket 117 2016-04-07T03:43:28.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33360 Milliseconds Vbucket 117 2016-04-07T03:43:28.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33390 Milliseconds Vbucket 117 2016-04-07T03:43:28.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33420 Milliseconds Vbucket 117 2016-04-07T03:43:28.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33450 Milliseconds Vbucket 117 2016-04-07T03:43:28.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33480 Milliseconds Vbucket 117 2016-04-07T03:43:28.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33510 Milliseconds Vbucket 117 2016-04-07T03:43:28.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33540 Milliseconds Vbucket 117 2016-04-07T03:43:28.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33570 Milliseconds Vbucket 117 2016-04-07T03:43:28.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33600 Milliseconds Vbucket 117 2016-04-07T03:43:28.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33630 Milliseconds Vbucket 117 2016-04-07T03:43:28.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33660 Milliseconds Vbucket 117 2016-04-07T03:43:28.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33690 Milliseconds Vbucket 117 2016-04-07T03:43:28.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33720 Milliseconds Vbucket 117 2016-04-07T03:43:28.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33750 Milliseconds Vbucket 117 2016-04-07T03:43:28.468-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 19.295µs CommitTime 372.498126ms TotalFlushTime 3m37.820715829s TotalCommitTime 9.430276349s 2016-04-07T03:43:28.468-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 6244395, 6244395, 40 committed:true 2016-04-07T03:43:28.468-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 6244395, 6244395, 40 committed:true) SnapCreateDur 385.269355ms SnapOpenDur 59.25µs 2016-04-07T03:43:28.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33780 Milliseconds Vbucket 117 2016-04-07T03:43:28.51-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:28.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33810 Milliseconds Vbucket 117 2016-04-07T03:43:28.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33840 Milliseconds Vbucket 117 2016-04-07T03:43:28.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33870 Milliseconds Vbucket 117 2016-04-07T03:43:28.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33900 Milliseconds Vbucket 117 2016-04-07T03:43:28.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33930 Milliseconds Vbucket 117 2016-04-07T03:43:28.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33960 Milliseconds Vbucket 117 2016-04-07T03:43:28.686-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6250000 QueuedCount 50000 2016-04-07T03:43:28.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33990 Milliseconds Vbucket 117 2016-04-07T03:43:28.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34020 Milliseconds Vbucket 117 2016-04-07T03:43:28.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34050 Milliseconds Vbucket 117 2016-04-07T03:43:28.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34080 Milliseconds Vbucket 117 2016-04-07T03:43:28.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34110 Milliseconds Vbucket 117 2016-04-07T03:43:28.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34140 Milliseconds Vbucket 117 2016-04-07T03:43:28.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34170 Milliseconds Vbucket 117 2016-04-07T03:43:28.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34200 Milliseconds Vbucket 117 2016-04-07T03:43:28.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34230 Milliseconds Vbucket 117 2016-04-07T03:43:28.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34260 Milliseconds Vbucket 117 2016-04-07T03:43:29.008-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34290 Milliseconds Vbucket 117 2016-04-07T03:43:29.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34320 Milliseconds Vbucket 117 2016-04-07T03:43:29.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34350 Milliseconds Vbucket 117 2016-04-07T03:43:29.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34380 Milliseconds Vbucket 117 2016-04-07T03:43:29.102-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6260000 QueuedCount 50000 2016-04-07T03:43:29.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34410 Milliseconds Vbucket 117 2016-04-07T03:43:29.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34440 Milliseconds Vbucket 117 2016-04-07T03:43:29.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34470 Milliseconds Vbucket 117 2016-04-07T03:43:29.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34500 Milliseconds Vbucket 117 2016-04-07T03:43:29.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34530 Milliseconds Vbucket 117 2016-04-07T03:43:29.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34560 Milliseconds Vbucket 117 2016-04-07T03:43:29.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34590 Milliseconds Vbucket 117 2016-04-07T03:43:29.335-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:29.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34620 Milliseconds Vbucket 117 2016-04-07T03:43:29.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34650 Milliseconds Vbucket 117 2016-04-07T03:43:29.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34680 Milliseconds Vbucket 117 2016-04-07T03:43:29.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34710 Milliseconds Vbucket 117 2016-04-07T03:43:29.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34740 Milliseconds Vbucket 117 2016-04-07T03:43:29.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34770 Milliseconds Vbucket 117 2016-04-07T03:43:29.501-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6270000 QueuedCount 50000 2016-04-07T03:43:29.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34800 Milliseconds Vbucket 117 2016-04-07T03:43:29.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34830 Milliseconds Vbucket 117 2016-04-07T03:43:29.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34860 Milliseconds Vbucket 117 2016-04-07T03:43:29.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34890 Milliseconds Vbucket 117 2016-04-07T03:43:29.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34920 Milliseconds Vbucket 117 2016-04-07T03:43:29.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34950 Milliseconds Vbucket 117 2016-04-07T03:43:29.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34980 Milliseconds Vbucket 117 2016-04-07T03:43:29.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35010 Milliseconds Vbucket 117 2016-04-07T03:43:29.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35040 Milliseconds Vbucket 117 2016-04-07T03:43:29.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35070 Milliseconds Vbucket 117 2016-04-07T03:43:29.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35100 Milliseconds Vbucket 117 2016-04-07T03:43:29.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35130 Milliseconds Vbucket 117 2016-04-07T03:43:29.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35160 Milliseconds Vbucket 117 2016-04-07T03:43:29.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35190 Milliseconds Vbucket 117 2016-04-07T03:43:29.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35220 Milliseconds Vbucket 117 2016-04-07T03:43:29.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35250 Milliseconds Vbucket 117 2016-04-07T03:43:29.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35280 Milliseconds Vbucket 117 2016-04-07T03:43:30.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35310 Milliseconds Vbucket 117 2016-04-07T03:43:30.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35340 Milliseconds Vbucket 117 2016-04-07T03:43:30.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35370 Milliseconds Vbucket 117 2016-04-07T03:43:30.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35400 Milliseconds Vbucket 117 2016-04-07T03:43:30.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35430 Milliseconds Vbucket 117 2016-04-07T03:43:30.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35460 Milliseconds Vbucket 117 2016-04-07T03:43:30.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35490 Milliseconds Vbucket 117 2016-04-07T03:43:30.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35520 Milliseconds Vbucket 117 2016-04-07T03:43:30.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35550 Milliseconds Vbucket 117 2016-04-07T03:43:30.283-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6280000 QueuedCount 50000 2016-04-07T03:43:30.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35580 Milliseconds Vbucket 117 2016-04-07T03:43:30.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35610 Milliseconds Vbucket 117 2016-04-07T03:43:30.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35640 Milliseconds Vbucket 117 2016-04-07T03:43:30.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35670 Milliseconds Vbucket 117 2016-04-07T03:43:30.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35700 Milliseconds Vbucket 117 2016-04-07T03:43:30.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35730 Milliseconds Vbucket 117 2016-04-07T03:43:30.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35760 Milliseconds Vbucket 117 2016-04-07T03:43:30.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35790 Milliseconds Vbucket 117 2016-04-07T03:43:30.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35820 Milliseconds Vbucket 117 2016-04-07T03:43:30.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35850 Milliseconds Vbucket 117 2016-04-07T03:43:30.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35880 Milliseconds Vbucket 117 2016-04-07T03:43:30.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35910 Milliseconds Vbucket 117 2016-04-07T03:43:30.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35940 Milliseconds Vbucket 117 2016-04-07T03:43:30.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35970 Milliseconds Vbucket 117 2016-04-07T03:43:30.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36000 Milliseconds Vbucket 117 2016-04-07T03:43:30.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36030 Milliseconds Vbucket 117 2016-04-07T03:43:30.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36060 Milliseconds Vbucket 117 2016-04-07T03:43:30.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36090 Milliseconds Vbucket 117 2016-04-07T03:43:30.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36120 Milliseconds Vbucket 117 2016-04-07T03:43:30.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36150 Milliseconds Vbucket 117 2016-04-07T03:43:30.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36180 Milliseconds Vbucket 117 2016-04-07T03:43:30.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36210 Milliseconds Vbucket 117 2016-04-07T03:43:30.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36240 Milliseconds Vbucket 117 2016-04-07T03:43:30.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36270 Milliseconds Vbucket 117 2016-04-07T03:43:31.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36300 Milliseconds Vbucket 117 2016-04-07T03:43:31.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36330 Milliseconds Vbucket 117 2016-04-07T03:43:31.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36360 Milliseconds Vbucket 117 2016-04-07T03:43:31.093-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6290000 QueuedCount 50000 2016-04-07T03:43:31.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36390 Milliseconds Vbucket 117 2016-04-07T03:43:31.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36420 Milliseconds Vbucket 117 2016-04-07T03:43:31.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36450 Milliseconds Vbucket 117 2016-04-07T03:43:31.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36480 Milliseconds Vbucket 117 2016-04-07T03:43:31.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36510 Milliseconds Vbucket 117 2016-04-07T03:43:31.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36540 Milliseconds Vbucket 117 2016-04-07T03:43:31.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36570 Milliseconds Vbucket 117 2016-04-07T03:43:31.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36600 Milliseconds Vbucket 117 2016-04-07T03:43:31.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36630 Milliseconds Vbucket 117 2016-04-07T03:43:31.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36660 Milliseconds Vbucket 117 2016-04-07T03:43:31.406-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:31.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36690 Milliseconds Vbucket 117 2016-04-07T03:43:31.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36720 Milliseconds Vbucket 117 2016-04-07T03:43:31.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36750 Milliseconds Vbucket 117 2016-04-07T03:43:31.474-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6300000 QueuedCount 50000 2016-04-07T03:43:31.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36780 Milliseconds Vbucket 117 2016-04-07T03:43:31.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36810 Milliseconds Vbucket 117 2016-04-07T03:43:31.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36840 Milliseconds Vbucket 117 2016-04-07T03:43:31.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36870 Milliseconds Vbucket 117 2016-04-07T03:43:31.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36900 Milliseconds Vbucket 117 2016-04-07T03:43:31.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36930 Milliseconds Vbucket 117 2016-04-07T03:43:31.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36960 Milliseconds Vbucket 117 2016-04-07T03:43:31.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36990 Milliseconds Vbucket 117 2016-04-07T03:43:31.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37020 Milliseconds Vbucket 117 2016-04-07T03:43:31.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37050 Milliseconds Vbucket 117 2016-04-07T03:43:31.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37080 Milliseconds Vbucket 117 2016-04-07T03:43:31.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37110 Milliseconds Vbucket 117 2016-04-07T03:43:31.837-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6310000 QueuedCount 50000 2016-04-07T03:43:31.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37140 Milliseconds Vbucket 117 2016-04-07T03:43:31.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37170 Milliseconds Vbucket 117 2016-04-07T03:43:31.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37200 Milliseconds Vbucket 117 2016-04-07T03:43:31.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37230 Milliseconds Vbucket 117 2016-04-07T03:43:31.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37260 Milliseconds Vbucket 117 2016-04-07T03:43:32.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37290 Milliseconds Vbucket 117 2016-04-07T03:43:32.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37320 Milliseconds Vbucket 117 2016-04-07T03:43:32.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37350 Milliseconds Vbucket 117 2016-04-07T03:43:32.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37380 Milliseconds Vbucket 117 2016-04-07T03:43:32.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37410 Milliseconds Vbucket 117 2016-04-07T03:43:32.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37440 Milliseconds Vbucket 117 2016-04-07T03:43:32.174-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6320000 QueuedCount 49976 2016-04-07T03:43:32.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37470 Milliseconds Vbucket 117 2016-04-07T03:43:32.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37500 Milliseconds Vbucket 117 2016-04-07T03:43:32.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37530 Milliseconds Vbucket 117 2016-04-07T03:43:32.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37560 Milliseconds Vbucket 117 2016-04-07T03:43:32.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37590 Milliseconds Vbucket 117 2016-04-07T03:43:32.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37620 Milliseconds Vbucket 117 2016-04-07T03:43:32.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37650 Milliseconds Vbucket 117 2016-04-07T03:43:32.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37680 Milliseconds Vbucket 117 2016-04-07T03:43:32.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37710 Milliseconds Vbucket 117 2016-04-07T03:43:32.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37740 Milliseconds Vbucket 117 2016-04-07T03:43:32.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37770 Milliseconds Vbucket 117 2016-04-07T03:43:32.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37800 Milliseconds Vbucket 117 2016-04-07T03:43:32.542-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6330000 QueuedCount 50000 2016-04-07T03:43:32.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37830 Milliseconds Vbucket 117 2016-04-07T03:43:32.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37860 Milliseconds Vbucket 117 2016-04-07T03:43:32.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37890 Milliseconds Vbucket 117 2016-04-07T03:43:32.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37920 Milliseconds Vbucket 117 2016-04-07T03:43:32.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37950 Milliseconds Vbucket 117 2016-04-07T03:43:32.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37980 Milliseconds Vbucket 117 2016-04-07T03:43:32.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38010 Milliseconds Vbucket 117 2016-04-07T03:43:32.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38040 Milliseconds Vbucket 117 2016-04-07T03:43:32.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38070 Milliseconds Vbucket 117 2016-04-07T03:43:32.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38100 Milliseconds Vbucket 117 2016-04-07T03:43:32.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38130 Milliseconds Vbucket 117 2016-04-07T03:43:32.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38160 Milliseconds Vbucket 117 2016-04-07T03:43:32.879-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6340000 QueuedCount 42034 2016-04-07T03:43:32.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38190 Milliseconds Vbucket 117 2016-04-07T03:43:32.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38220 Milliseconds Vbucket 117 2016-04-07T03:43:32.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38250 Milliseconds Vbucket 117 2016-04-07T03:43:32.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38280 Milliseconds Vbucket 117 2016-04-07T03:43:33.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38310 Milliseconds Vbucket 117 2016-04-07T03:43:33.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38340 Milliseconds Vbucket 117 2016-04-07T03:43:33.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38370 Milliseconds Vbucket 117 2016-04-07T03:43:33.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38400 Milliseconds Vbucket 117 2016-04-07T03:43:33.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38430 Milliseconds Vbucket 117 2016-04-07T03:43:33.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38460 Milliseconds Vbucket 117 2016-04-07T03:43:33.201-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6350000 QueuedCount 32034 2016-04-07T03:43:33.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38490 Milliseconds Vbucket 117 2016-04-07T03:43:33.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38520 Milliseconds Vbucket 117 2016-04-07T03:43:33.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38550 Milliseconds Vbucket 117 2016-04-07T03:43:33.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38580 Milliseconds Vbucket 117 2016-04-07T03:43:33.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38610 Milliseconds Vbucket 117 2016-04-07T03:43:33.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38640 Milliseconds Vbucket 117 2016-04-07T03:43:33.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38670 Milliseconds Vbucket 117 2016-04-07T03:43:33.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38700 Milliseconds Vbucket 117 2016-04-07T03:43:33.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38730 Milliseconds Vbucket 117 2016-04-07T03:43:33.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38760 Milliseconds Vbucket 117 2016-04-07T03:43:33.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38790 Milliseconds Vbucket 117 2016-04-07T03:43:33.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38820 Milliseconds Vbucket 117 2016-04-07T03:43:33.545-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6360000 QueuedCount 22034 2016-04-07T03:43:33.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38850 Milliseconds Vbucket 117 2016-04-07T03:43:33.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38880 Milliseconds Vbucket 117 2016-04-07T03:43:33.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38910 Milliseconds Vbucket 117 2016-04-07T03:43:33.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38940 Milliseconds Vbucket 117 2016-04-07T03:43:33.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38970 Milliseconds Vbucket 117 2016-04-07T03:43:33.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39000 Milliseconds Vbucket 117 2016-04-07T03:43:33.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39030 Milliseconds Vbucket 117 2016-04-07T03:43:33.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39060 Milliseconds Vbucket 117 2016-04-07T03:43:33.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39090 Milliseconds Vbucket 117 2016-04-07T03:43:33.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39120 Milliseconds Vbucket 117 2016-04-07T03:43:33.859-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6370000 QueuedCount 12034 2016-04-07T03:43:33.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39150 Milliseconds Vbucket 117 2016-04-07T03:43:33.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39180 Milliseconds Vbucket 117 2016-04-07T03:43:33.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39210 Milliseconds Vbucket 117 2016-04-07T03:43:33.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39240 Milliseconds Vbucket 117 2016-04-07T03:43:33.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39270 Milliseconds Vbucket 117 2016-04-07T03:43:34.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39300 Milliseconds Vbucket 117 2016-04-07T03:43:34.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39330 Milliseconds Vbucket 117 2016-04-07T03:43:34.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39360 Milliseconds Vbucket 117 2016-04-07T03:43:34.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39390 Milliseconds Vbucket 117 2016-04-07T03:43:34.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39420 Milliseconds Vbucket 117 2016-04-07T03:43:34.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39450 Milliseconds Vbucket 117 2016-04-07T03:43:34.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39480 Milliseconds Vbucket 117 2016-04-07T03:43:34.2-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6380000 QueuedCount 2034 2016-04-07T03:43:34.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39510 Milliseconds Vbucket 117 2016-04-07T03:43:34.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39540 Milliseconds Vbucket 117 2016-04-07T03:43:34.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39570 Milliseconds Vbucket 117 2016-04-07T03:43:34.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39600 Milliseconds Vbucket 117 2016-04-07T03:43:34.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39630 Milliseconds Vbucket 117 2016-04-07T03:43:34.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39660 Milliseconds Vbucket 117 2016-04-07T03:43:34.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39690 Milliseconds Vbucket 117 2016-04-07T03:43:34.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39720 Milliseconds Vbucket 117 2016-04-07T03:43:34.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39750 Milliseconds Vbucket 117 2016-04-07T03:43:34.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39780 Milliseconds Vbucket 117 2016-04-07T03:43:34.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39810 Milliseconds Vbucket 117 2016-04-07T03:43:34.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39840 Milliseconds Vbucket 117 2016-04-07T03:43:34.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39870 Milliseconds Vbucket 117 2016-04-07T03:43:34.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39900 Milliseconds Vbucket 117 2016-04-07T03:43:34.63-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.621µs CommitTime 344.973119ms TotalFlushTime 3m43.390380521s TotalCommitTime 9.775249468s 2016-04-07T03:43:34.63-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 6382034, 6382034, 41 committed:true 2016-04-07T03:43:34.63-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 6382034, 6382034, 41 committed:true) SnapCreateDur 355.847179ms SnapOpenDur 46.641µs 2016-04-07T03:43:34.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39930 Milliseconds Vbucket 117 2016-04-07T03:43:34.671-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:34.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39960 Milliseconds Vbucket 117 2016-04-07T03:43:34.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39990 Milliseconds Vbucket 117 2016-04-07T03:43:34.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40020 Milliseconds Vbucket 117 2016-04-07T03:43:34.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40050 Milliseconds Vbucket 117 2016-04-07T03:43:34.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40080 Milliseconds Vbucket 117 2016-04-07T03:43:34.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40110 Milliseconds Vbucket 117 2016-04-07T03:43:34.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40140 Milliseconds Vbucket 117 2016-04-07T03:43:34.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40170 Milliseconds Vbucket 117 2016-04-07T03:43:34.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40200 Milliseconds Vbucket 117 2016-04-07T03:43:34.943-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6390000 QueuedCount 50000 2016-04-07T03:43:34.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40230 Milliseconds Vbucket 117 2016-04-07T03:43:34.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40260 Milliseconds Vbucket 117 2016-04-07T03:43:35.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40290 Milliseconds Vbucket 117 2016-04-07T03:43:35.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40320 Milliseconds Vbucket 117 2016-04-07T03:43:35.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40350 Milliseconds Vbucket 117 2016-04-07T03:43:35.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40380 Milliseconds Vbucket 117 2016-04-07T03:43:35.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40410 Milliseconds Vbucket 117 2016-04-07T03:43:35.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40440 Milliseconds Vbucket 117 2016-04-07T03:43:35.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40470 Milliseconds Vbucket 117 2016-04-07T03:43:35.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40500 Milliseconds Vbucket 117 2016-04-07T03:43:35.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40530 Milliseconds Vbucket 117 2016-04-07T03:43:35.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40560 Milliseconds Vbucket 117 2016-04-07T03:43:35.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40590 Milliseconds Vbucket 117 2016-04-07T03:43:35.328-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:35.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40620 Milliseconds Vbucket 117 2016-04-07T03:43:35.345-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6400000 QueuedCount 48581 2016-04-07T03:43:35.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40650 Milliseconds Vbucket 117 2016-04-07T03:43:35.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40680 Milliseconds Vbucket 117 2016-04-07T03:43:35.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40710 Milliseconds Vbucket 117 2016-04-07T03:43:35.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40740 Milliseconds Vbucket 117 2016-04-07T03:43:35.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40770 Milliseconds Vbucket 117 2016-04-07T03:43:35.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40800 Milliseconds Vbucket 117 2016-04-07T03:43:35.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40830 Milliseconds Vbucket 117 2016-04-07T03:43:35.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40860 Milliseconds Vbucket 117 2016-04-07T03:43:35.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40890 Milliseconds Vbucket 117 2016-04-07T03:43:35.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40920 Milliseconds Vbucket 117 2016-04-07T03:43:35.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40950 Milliseconds Vbucket 117 2016-04-07T03:43:35.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40980 Milliseconds Vbucket 117 2016-04-07T03:43:35.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41010 Milliseconds Vbucket 117 2016-04-07T03:43:35.732-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6410000 QueuedCount 50000 2016-04-07T03:43:35.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41040 Milliseconds Vbucket 117 2016-04-07T03:43:35.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41070 Milliseconds Vbucket 117 2016-04-07T03:43:35.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41100 Milliseconds Vbucket 117 2016-04-07T03:43:35.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41130 Milliseconds Vbucket 117 2016-04-07T03:43:35.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41160 Milliseconds Vbucket 117 2016-04-07T03:43:35.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41190 Milliseconds Vbucket 117 2016-04-07T03:43:35.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41220 Milliseconds Vbucket 117 2016-04-07T03:43:35.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41250 Milliseconds Vbucket 117 2016-04-07T03:43:35.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41280 Milliseconds Vbucket 117 2016-04-07T03:43:36.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41310 Milliseconds Vbucket 117 2016-04-07T03:43:36.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41340 Milliseconds Vbucket 117 2016-04-07T03:43:36.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41370 Milliseconds Vbucket 117 2016-04-07T03:43:36.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41400 Milliseconds Vbucket 117 2016-04-07T03:43:36.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41430 Milliseconds Vbucket 117 2016-04-07T03:43:36.153-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6420000 QueuedCount 50000 2016-04-07T03:43:36.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41460 Milliseconds Vbucket 117 2016-04-07T03:43:36.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41490 Milliseconds Vbucket 117 2016-04-07T03:43:36.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41520 Milliseconds Vbucket 117 2016-04-07T03:43:36.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41550 Milliseconds Vbucket 117 2016-04-07T03:43:36.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41580 Milliseconds Vbucket 117 2016-04-07T03:43:36.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41610 Milliseconds Vbucket 117 2016-04-07T03:43:36.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41640 Milliseconds Vbucket 117 2016-04-07T03:43:36.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41670 Milliseconds Vbucket 117 2016-04-07T03:43:36.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41700 Milliseconds Vbucket 117 2016-04-07T03:43:36.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41730 Milliseconds Vbucket 117 2016-04-07T03:43:36.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41760 Milliseconds Vbucket 117 2016-04-07T03:43:36.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41790 Milliseconds Vbucket 117 2016-04-07T03:43:36.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41820 Milliseconds Vbucket 117 2016-04-07T03:43:36.544-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6430000 QueuedCount 50000 2016-04-07T03:43:36.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41850 Milliseconds Vbucket 117 2016-04-07T03:43:36.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41880 Milliseconds Vbucket 117 2016-04-07T03:43:36.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41910 Milliseconds Vbucket 117 2016-04-07T03:43:36.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41940 Milliseconds Vbucket 117 2016-04-07T03:43:36.681-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:36.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41970 Milliseconds Vbucket 117 2016-04-07T03:43:36.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42000 Milliseconds Vbucket 117 2016-04-07T03:43:36.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42030 Milliseconds Vbucket 117 2016-04-07T03:43:36.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42060 Milliseconds Vbucket 117 2016-04-07T03:43:36.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42090 Milliseconds Vbucket 117 2016-04-07T03:43:36.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42120 Milliseconds Vbucket 117 2016-04-07T03:43:36.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42150 Milliseconds Vbucket 117 2016-04-07T03:43:36.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42180 Milliseconds Vbucket 117 2016-04-07T03:43:36.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42210 Milliseconds Vbucket 117 2016-04-07T03:43:36.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42240 Milliseconds Vbucket 117 2016-04-07T03:43:36.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42270 Milliseconds Vbucket 117 2016-04-07T03:43:37.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42300 Milliseconds Vbucket 117 2016-04-07T03:43:37.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42330 Milliseconds Vbucket 117 2016-04-07T03:43:37.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42360 Milliseconds Vbucket 117 2016-04-07T03:43:37.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42390 Milliseconds Vbucket 117 2016-04-07T03:43:37.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42420 Milliseconds Vbucket 117 2016-04-07T03:43:37.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42450 Milliseconds Vbucket 117 2016-04-07T03:43:37.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42480 Milliseconds Vbucket 117 2016-04-07T03:43:37.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42510 Milliseconds Vbucket 117 2016-04-07T03:43:37.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42540 Milliseconds Vbucket 117 2016-04-07T03:43:37.288-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42570 Milliseconds Vbucket 117 2016-04-07T03:43:37.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42600 Milliseconds Vbucket 117 2016-04-07T03:43:37.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42630 Milliseconds Vbucket 117 2016-04-07T03:43:37.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42660 Milliseconds Vbucket 117 2016-04-07T03:43:37.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42690 Milliseconds Vbucket 117 2016-04-07T03:43:37.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42720 Milliseconds Vbucket 117 2016-04-07T03:43:37.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42750 Milliseconds Vbucket 117 2016-04-07T03:43:37.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42780 Milliseconds Vbucket 117 2016-04-07T03:43:37.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42810 Milliseconds Vbucket 117 2016-04-07T03:43:37.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42840 Milliseconds Vbucket 117 2016-04-07T03:43:37.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42870 Milliseconds Vbucket 117 2016-04-07T03:43:37.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42900 Milliseconds Vbucket 117 2016-04-07T03:43:37.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42930 Milliseconds Vbucket 117 2016-04-07T03:43:37.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42960 Milliseconds Vbucket 117 2016-04-07T03:43:37.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42990 Milliseconds Vbucket 117 2016-04-07T03:43:37.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43020 Milliseconds Vbucket 117 2016-04-07T03:43:37.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43050 Milliseconds Vbucket 117 2016-04-07T03:43:37.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43080 Milliseconds Vbucket 117 2016-04-07T03:43:37.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43110 Milliseconds Vbucket 117 2016-04-07T03:43:37.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43140 Milliseconds Vbucket 117 2016-04-07T03:43:37.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43170 Milliseconds Vbucket 117 2016-04-07T03:43:37.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43200 Milliseconds Vbucket 117 2016-04-07T03:43:37.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43230 Milliseconds Vbucket 117 2016-04-07T03:43:37.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43260 Milliseconds Vbucket 117 2016-04-07T03:43:38.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43290 Milliseconds Vbucket 117 2016-04-07T03:43:38.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43320 Milliseconds Vbucket 117 2016-04-07T03:43:38.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43350 Milliseconds Vbucket 117 2016-04-07T03:43:38.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43380 Milliseconds Vbucket 117 2016-04-07T03:43:38.119-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6440000 QueuedCount 50000 2016-04-07T03:43:38.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43410 Milliseconds Vbucket 117 2016-04-07T03:43:38.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43440 Milliseconds Vbucket 117 2016-04-07T03:43:38.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43470 Milliseconds Vbucket 117 2016-04-07T03:43:38.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43500 Milliseconds Vbucket 117 2016-04-07T03:43:38.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43530 Milliseconds Vbucket 117 2016-04-07T03:43:38.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43560 Milliseconds Vbucket 117 2016-04-07T03:43:38.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43590 Milliseconds Vbucket 117 2016-04-07T03:43:38.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43620 Milliseconds Vbucket 117 2016-04-07T03:43:38.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43650 Milliseconds Vbucket 117 2016-04-07T03:43:38.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43680 Milliseconds Vbucket 117 2016-04-07T03:43:38.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43710 Milliseconds Vbucket 117 2016-04-07T03:43:38.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43740 Milliseconds Vbucket 117 2016-04-07T03:43:38.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43770 Milliseconds Vbucket 117 2016-04-07T03:43:38.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43800 Milliseconds Vbucket 117 2016-04-07T03:43:38.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43830 Milliseconds Vbucket 117 2016-04-07T03:43:38.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43860 Milliseconds Vbucket 117 2016-04-07T03:43:38.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43890 Milliseconds Vbucket 117 2016-04-07T03:43:38.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43920 Milliseconds Vbucket 117 2016-04-07T03:43:38.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43950 Milliseconds Vbucket 117 2016-04-07T03:43:38.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43980 Milliseconds Vbucket 117 2016-04-07T03:43:38.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44010 Milliseconds Vbucket 117 2016-04-07T03:43:38.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44040 Milliseconds Vbucket 117 2016-04-07T03:43:38.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44070 Milliseconds Vbucket 117 2016-04-07T03:43:38.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44100 Milliseconds Vbucket 117 2016-04-07T03:43:38.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44130 Milliseconds Vbucket 117 2016-04-07T03:43:38.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44160 Milliseconds Vbucket 117 2016-04-07T03:43:38.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44190 Milliseconds Vbucket 117 2016-04-07T03:43:38.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44220 Milliseconds Vbucket 117 2016-04-07T03:43:38.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44250 Milliseconds Vbucket 117 2016-04-07T03:43:38.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44280 Milliseconds Vbucket 117 2016-04-07T03:43:39.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44310 Milliseconds Vbucket 117 2016-04-07T03:43:39.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44340 Milliseconds Vbucket 117 2016-04-07T03:43:39.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44370 Milliseconds Vbucket 117 2016-04-07T03:43:39.087-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6450000 QueuedCount 50000 2016-04-07T03:43:39.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44400 Milliseconds Vbucket 117 2016-04-07T03:43:39.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44430 Milliseconds Vbucket 117 2016-04-07T03:43:39.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44460 Milliseconds Vbucket 117 2016-04-07T03:43:39.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44490 Milliseconds Vbucket 117 2016-04-07T03:43:39.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44520 Milliseconds Vbucket 117 2016-04-07T03:43:39.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44550 Milliseconds Vbucket 117 2016-04-07T03:43:39.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44580 Milliseconds Vbucket 117 2016-04-07T03:43:39.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44610 Milliseconds Vbucket 117 2016-04-07T03:43:39.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44640 Milliseconds Vbucket 117 2016-04-07T03:43:39.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44670 Milliseconds Vbucket 117 2016-04-07T03:43:39.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44700 Milliseconds Vbucket 117 2016-04-07T03:43:39.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44730 Milliseconds Vbucket 117 2016-04-07T03:43:39.467-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6460000 QueuedCount 50000 2016-04-07T03:43:39.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44760 Milliseconds Vbucket 117 2016-04-07T03:43:39.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44790 Milliseconds Vbucket 117 2016-04-07T03:43:39.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44820 Milliseconds Vbucket 117 2016-04-07T03:43:39.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44850 Milliseconds Vbucket 117 2016-04-07T03:43:39.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44880 Milliseconds Vbucket 117 2016-04-07T03:43:39.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44910 Milliseconds Vbucket 117 2016-04-07T03:43:39.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44940 Milliseconds Vbucket 117 2016-04-07T03:43:39.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44970 Milliseconds Vbucket 117 2016-04-07T03:43:39.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45000 Milliseconds Vbucket 117 2016-04-07T03:43:39.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45030 Milliseconds Vbucket 117 2016-04-07T03:43:39.763-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6470000 QueuedCount 44674 2016-04-07T03:43:39.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45060 Milliseconds Vbucket 117 2016-04-07T03:43:39.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45090 Milliseconds Vbucket 117 2016-04-07T03:43:39.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45120 Milliseconds Vbucket 117 2016-04-07T03:43:39.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45150 Milliseconds Vbucket 117 2016-04-07T03:43:39.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45180 Milliseconds Vbucket 117 2016-04-07T03:43:39.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45210 Milliseconds Vbucket 117 2016-04-07T03:43:39.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45240 Milliseconds Vbucket 117 2016-04-07T03:43:39.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45270 Milliseconds Vbucket 117 2016-04-07T03:43:40.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45300 Milliseconds Vbucket 117 2016-04-07T03:43:40.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45330 Milliseconds Vbucket 117 2016-04-07T03:43:40.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45360 Milliseconds Vbucket 117 2016-04-07T03:43:40.09-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6480000 QueuedCount 34674 2016-04-07T03:43:40.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45390 Milliseconds Vbucket 117 2016-04-07T03:43:40.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45420 Milliseconds Vbucket 117 2016-04-07T03:43:40.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45450 Milliseconds Vbucket 117 2016-04-07T03:43:40.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45480 Milliseconds Vbucket 117 2016-04-07T03:43:40.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45510 Milliseconds Vbucket 117 2016-04-07T03:43:40.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45540 Milliseconds Vbucket 117 2016-04-07T03:43:40.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45570 Milliseconds Vbucket 117 2016-04-07T03:43:40.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45600 Milliseconds Vbucket 117 2016-04-07T03:43:40.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45630 Milliseconds Vbucket 117 2016-04-07T03:43:40.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45660 Milliseconds Vbucket 117 2016-04-07T03:43:40.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45690 Milliseconds Vbucket 117 2016-04-07T03:43:40.415-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6490000 QueuedCount 24674 2016-04-07T03:43:40.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45720 Milliseconds Vbucket 117 2016-04-07T03:43:40.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45750 Milliseconds Vbucket 117 2016-04-07T03:43:40.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45780 Milliseconds Vbucket 117 2016-04-07T03:43:40.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45810 Milliseconds Vbucket 117 2016-04-07T03:43:40.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45840 Milliseconds Vbucket 117 2016-04-07T03:43:40.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45870 Milliseconds Vbucket 117 2016-04-07T03:43:40.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45900 Milliseconds Vbucket 117 2016-04-07T03:43:40.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45930 Milliseconds Vbucket 117 2016-04-07T03:43:40.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45960 Milliseconds Vbucket 117 2016-04-07T03:43:40.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45990 Milliseconds Vbucket 117 2016-04-07T03:43:40.73-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6500000 QueuedCount 14674 2016-04-07T03:43:40.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46020 Milliseconds Vbucket 117 2016-04-07T03:43:40.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46050 Milliseconds Vbucket 117 2016-04-07T03:43:40.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46080 Milliseconds Vbucket 117 2016-04-07T03:43:40.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46110 Milliseconds Vbucket 117 2016-04-07T03:43:40.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46140 Milliseconds Vbucket 117 2016-04-07T03:43:40.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46170 Milliseconds Vbucket 117 2016-04-07T03:43:40.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46200 Milliseconds Vbucket 117 2016-04-07T03:43:40.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46230 Milliseconds Vbucket 117 2016-04-07T03:43:40.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46260 Milliseconds Vbucket 117 2016-04-07T03:43:41.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46290 Milliseconds Vbucket 117 2016-04-07T03:43:41.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46320 Milliseconds Vbucket 117 2016-04-07T03:43:41.054-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6510000 QueuedCount 4674 2016-04-07T03:43:41.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46350 Milliseconds Vbucket 117 2016-04-07T03:43:41.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46380 Milliseconds Vbucket 117 2016-04-07T03:43:41.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46410 Milliseconds Vbucket 117 2016-04-07T03:43:41.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46440 Milliseconds Vbucket 117 2016-04-07T03:43:41.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46470 Milliseconds Vbucket 117 2016-04-07T03:43:41.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46500 Milliseconds Vbucket 117 2016-04-07T03:43:41.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46530 Milliseconds Vbucket 117 2016-04-07T03:43:41.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46560 Milliseconds Vbucket 117 2016-04-07T03:43:41.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46590 Milliseconds Vbucket 117 2016-04-07T03:43:41.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46620 Milliseconds Vbucket 117 2016-04-07T03:43:41.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46650 Milliseconds Vbucket 117 2016-04-07T03:43:41.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46680 Milliseconds Vbucket 117 2016-04-07T03:43:41.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46710 Milliseconds Vbucket 117 2016-04-07T03:43:41.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46740 Milliseconds Vbucket 117 2016-04-07T03:43:41.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46770 Milliseconds Vbucket 117 2016-04-07T03:43:41.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46800 Milliseconds Vbucket 117 2016-04-07T03:43:41.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46830 Milliseconds Vbucket 117 2016-04-07T03:43:41.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46860 Milliseconds Vbucket 117 2016-04-07T03:43:41.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46890 Milliseconds Vbucket 117 2016-04-07T03:43:41.631-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.273µs CommitTime 429.97163ms TotalFlushTime 3m49.732668203s TotalCommitTime 10.205221098s 2016-04-07T03:43:41.631-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 6514674, 6514674, 42 committed:true 2016-04-07T03:43:41.631-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 6514674, 6514674, 42 committed:true) SnapCreateDur 440.15735ms SnapOpenDur 42.972µs 2016-04-07T03:43:41.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46920 Milliseconds Vbucket 117 2016-04-07T03:43:41.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46950 Milliseconds Vbucket 117 2016-04-07T03:43:41.67-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:41.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46980 Milliseconds Vbucket 117 2016-04-07T03:43:41.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47010 Milliseconds Vbucket 117 2016-04-07T03:43:41.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47040 Milliseconds Vbucket 117 2016-04-07T03:43:41.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47070 Milliseconds Vbucket 117 2016-04-07T03:43:41.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47100 Milliseconds Vbucket 117 2016-04-07T03:43:41.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47130 Milliseconds Vbucket 117 2016-04-07T03:43:41.866-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6520000 QueuedCount 50000 2016-04-07T03:43:41.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47160 Milliseconds Vbucket 117 2016-04-07T03:43:41.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47190 Milliseconds Vbucket 117 2016-04-07T03:43:41.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47220 Milliseconds Vbucket 117 2016-04-07T03:43:41.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47250 Milliseconds Vbucket 117 2016-04-07T03:43:41.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47280 Milliseconds Vbucket 117 2016-04-07T03:43:42.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47310 Milliseconds Vbucket 117 2016-04-07T03:43:42.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47340 Milliseconds Vbucket 117 2016-04-07T03:43:42.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47370 Milliseconds Vbucket 117 2016-04-07T03:43:42.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47400 Milliseconds Vbucket 117 2016-04-07T03:43:42.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47430 Milliseconds Vbucket 117 2016-04-07T03:43:42.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47460 Milliseconds Vbucket 117 2016-04-07T03:43:42.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47490 Milliseconds Vbucket 117 2016-04-07T03:43:42.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47520 Milliseconds Vbucket 117 2016-04-07T03:43:42.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47550 Milliseconds Vbucket 117 2016-04-07T03:43:42.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47580 Milliseconds Vbucket 117 2016-04-07T03:43:42.301-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6530000 QueuedCount 50000 2016-04-07T03:43:42.312-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:42.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47610 Milliseconds Vbucket 117 2016-04-07T03:43:42.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47640 Milliseconds Vbucket 117 2016-04-07T03:43:42.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47670 Milliseconds Vbucket 117 2016-04-07T03:43:42.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47700 Milliseconds Vbucket 117 2016-04-07T03:43:42.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47730 Milliseconds Vbucket 117 2016-04-07T03:43:42.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47760 Milliseconds Vbucket 117 2016-04-07T03:43:42.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47790 Milliseconds Vbucket 117 2016-04-07T03:43:42.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47820 Milliseconds Vbucket 117 2016-04-07T03:43:42.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47850 Milliseconds Vbucket 117 2016-04-07T03:43:42.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47880 Milliseconds Vbucket 117 2016-04-07T03:43:42.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47910 Milliseconds Vbucket 117 2016-04-07T03:43:42.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47940 Milliseconds Vbucket 117 2016-04-07T03:43:42.678-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6540000 QueuedCount 50000 2016-04-07T03:43:42.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47970 Milliseconds Vbucket 117 2016-04-07T03:43:42.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48000 Milliseconds Vbucket 117 2016-04-07T03:43:42.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48030 Milliseconds Vbucket 117 2016-04-07T03:43:42.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48060 Milliseconds Vbucket 117 2016-04-07T03:43:42.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48090 Milliseconds Vbucket 117 2016-04-07T03:43:42.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48120 Milliseconds Vbucket 117 2016-04-07T03:43:42.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48150 Milliseconds Vbucket 117 2016-04-07T03:43:42.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48180 Milliseconds Vbucket 117 2016-04-07T03:43:42.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48210 Milliseconds Vbucket 117 2016-04-07T03:43:42.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48240 Milliseconds Vbucket 117 2016-04-07T03:43:42.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48270 Milliseconds Vbucket 117 2016-04-07T03:43:43.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48300 Milliseconds Vbucket 117 2016-04-07T03:43:43.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48330 Milliseconds Vbucket 117 2016-04-07T03:43:43.049-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6550000 QueuedCount 50000 2016-04-07T03:43:43.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48360 Milliseconds Vbucket 117 2016-04-07T03:43:43.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48390 Milliseconds Vbucket 117 2016-04-07T03:43:43.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48420 Milliseconds Vbucket 117 2016-04-07T03:43:43.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48450 Milliseconds Vbucket 117 2016-04-07T03:43:43.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48480 Milliseconds Vbucket 117 2016-04-07T03:43:43.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48510 Milliseconds Vbucket 117 2016-04-07T03:43:43.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48540 Milliseconds Vbucket 117 2016-04-07T03:43:43.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48570 Milliseconds Vbucket 117 2016-04-07T03:43:43.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48600 Milliseconds Vbucket 117 2016-04-07T03:43:43.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48630 Milliseconds Vbucket 117 2016-04-07T03:43:43.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48660 Milliseconds Vbucket 117 2016-04-07T03:43:43.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48690 Milliseconds Vbucket 117 2016-04-07T03:43:43.415-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6560000 QueuedCount 50000 2016-04-07T03:43:43.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48720 Milliseconds Vbucket 117 2016-04-07T03:43:43.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48750 Milliseconds Vbucket 117 2016-04-07T03:43:43.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48780 Milliseconds Vbucket 117 2016-04-07T03:43:43.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48810 Milliseconds Vbucket 117 2016-04-07T03:43:43.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48840 Milliseconds Vbucket 117 2016-04-07T03:43:43.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48870 Milliseconds Vbucket 117 2016-04-07T03:43:43.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48900 Milliseconds Vbucket 117 2016-04-07T03:43:43.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48930 Milliseconds Vbucket 117 2016-04-07T03:43:43.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48960 Milliseconds Vbucket 117 2016-04-07T03:43:43.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48990 Milliseconds Vbucket 117 2016-04-07T03:43:43.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49020 Milliseconds Vbucket 117 2016-04-07T03:43:43.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49050 Milliseconds Vbucket 117 2016-04-07T03:43:43.772-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6570000 QueuedCount 50000 2016-04-07T03:43:43.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49080 Milliseconds Vbucket 117 2016-04-07T03:43:43.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49110 Milliseconds Vbucket 117 2016-04-07T03:43:43.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49140 Milliseconds Vbucket 117 2016-04-07T03:43:43.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49170 Milliseconds Vbucket 117 2016-04-07T03:43:43.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49200 Milliseconds Vbucket 117 2016-04-07T03:43:43.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49230 Milliseconds Vbucket 117 2016-04-07T03:43:43.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49260 Milliseconds Vbucket 117 2016-04-07T03:43:44.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49290 Milliseconds Vbucket 117 2016-04-07T03:43:44.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49320 Milliseconds Vbucket 117 2016-04-07T03:43:44.056-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6580000 QueuedCount 43382 2016-04-07T03:43:44.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49350 Milliseconds Vbucket 117 2016-04-07T03:43:44.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49380 Milliseconds Vbucket 117 2016-04-07T03:43:44.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49410 Milliseconds Vbucket 117 2016-04-07T03:43:44.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49440 Milliseconds Vbucket 117 2016-04-07T03:43:44.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49470 Milliseconds Vbucket 117 2016-04-07T03:43:44.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49500 Milliseconds Vbucket 117 2016-04-07T03:43:44.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49530 Milliseconds Vbucket 117 2016-04-07T03:43:44.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49560 Milliseconds Vbucket 117 2016-04-07T03:43:44.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49590 Milliseconds Vbucket 117 2016-04-07T03:43:44.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49620 Milliseconds Vbucket 117 2016-04-07T03:43:44.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49650 Milliseconds Vbucket 117 2016-04-07T03:43:44.391-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6590000 QueuedCount 33382 2016-04-07T03:43:44.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49680 Milliseconds Vbucket 117 2016-04-07T03:43:44.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49710 Milliseconds Vbucket 117 2016-04-07T03:43:44.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49740 Milliseconds Vbucket 117 2016-04-07T03:43:44.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49770 Milliseconds Vbucket 117 2016-04-07T03:43:44.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49800 Milliseconds Vbucket 117 2016-04-07T03:43:44.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49830 Milliseconds Vbucket 117 2016-04-07T03:43:44.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49860 Milliseconds Vbucket 117 2016-04-07T03:43:44.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49890 Milliseconds Vbucket 117 2016-04-07T03:43:44.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49920 Milliseconds Vbucket 117 2016-04-07T03:43:44.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49950 Milliseconds Vbucket 117 2016-04-07T03:43:44.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49980 Milliseconds Vbucket 117 2016-04-07T03:43:44.724-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6600000 QueuedCount 23382 2016-04-07T03:43:44.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50010 Milliseconds Vbucket 117 2016-04-07T03:43:44.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50040 Milliseconds Vbucket 117 2016-04-07T03:43:44.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50070 Milliseconds Vbucket 117 2016-04-07T03:43:44.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50100 Milliseconds Vbucket 117 2016-04-07T03:43:44.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50130 Milliseconds Vbucket 117 2016-04-07T03:43:44.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50160 Milliseconds Vbucket 117 2016-04-07T03:43:44.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50190 Milliseconds Vbucket 117 2016-04-07T03:43:44.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50220 Milliseconds Vbucket 117 2016-04-07T03:43:44.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50250 Milliseconds Vbucket 117 2016-04-07T03:43:44.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50280 Milliseconds Vbucket 117 2016-04-07T03:43:45.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50310 Milliseconds Vbucket 117 2016-04-07T03:43:45.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50340 Milliseconds Vbucket 117 2016-04-07T03:43:45.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50370 Milliseconds Vbucket 117 2016-04-07T03:43:45.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50400 Milliseconds Vbucket 117 2016-04-07T03:43:45.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50430 Milliseconds Vbucket 117 2016-04-07T03:43:45.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50460 Milliseconds Vbucket 117 2016-04-07T03:43:45.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50490 Milliseconds Vbucket 117 2016-04-07T03:43:45.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50520 Milliseconds Vbucket 117 2016-04-07T03:43:45.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50550 Milliseconds Vbucket 117 2016-04-07T03:43:45.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50580 Milliseconds Vbucket 117 2016-04-07T03:43:45.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50610 Milliseconds Vbucket 117 2016-04-07T03:43:45.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50640 Milliseconds Vbucket 117 2016-04-07T03:43:45.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50670 Milliseconds Vbucket 117 2016-04-07T03:43:45.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50700 Milliseconds Vbucket 117 2016-04-07T03:43:45.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50730 Milliseconds Vbucket 117 2016-04-07T03:43:45.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50760 Milliseconds Vbucket 117 2016-04-07T03:43:45.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50790 Milliseconds Vbucket 117 2016-04-07T03:43:45.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50820 Milliseconds Vbucket 117 2016-04-07T03:43:45.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50850 Milliseconds Vbucket 117 2016-04-07T03:43:45.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50880 Milliseconds Vbucket 117 2016-04-07T03:43:45.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50910 Milliseconds Vbucket 117 2016-04-07T03:43:45.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50940 Milliseconds Vbucket 117 2016-04-07T03:43:45.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50970 Milliseconds Vbucket 117 2016-04-07T03:43:45.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51000 Milliseconds Vbucket 117 2016-04-07T03:43:45.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51030 Milliseconds Vbucket 117 2016-04-07T03:43:45.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51060 Milliseconds Vbucket 117 2016-04-07T03:43:45.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51090 Milliseconds Vbucket 117 2016-04-07T03:43:45.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51120 Milliseconds Vbucket 117 2016-04-07T03:43:45.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51150 Milliseconds Vbucket 117 2016-04-07T03:43:45.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51180 Milliseconds Vbucket 117 2016-04-07T03:43:45.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51210 Milliseconds Vbucket 117 2016-04-07T03:43:45.931-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6610000 QueuedCount 13382 2016-04-07T03:43:45.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51240 Milliseconds Vbucket 117 2016-04-07T03:43:45.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51270 Milliseconds Vbucket 117 2016-04-07T03:43:46.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51300 Milliseconds Vbucket 117 2016-04-07T03:43:46.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51330 Milliseconds Vbucket 117 2016-04-07T03:43:46.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51360 Milliseconds Vbucket 117 2016-04-07T03:43:46.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51390 Milliseconds Vbucket 117 2016-04-07T03:43:46.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51420 Milliseconds Vbucket 117 2016-04-07T03:43:46.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51450 Milliseconds Vbucket 117 2016-04-07T03:43:46.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51480 Milliseconds Vbucket 117 2016-04-07T03:43:46.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51510 Milliseconds Vbucket 117 2016-04-07T03:43:46.228-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6620000 QueuedCount 3382 2016-04-07T03:43:46.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51540 Milliseconds Vbucket 117 2016-04-07T03:43:46.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51570 Milliseconds Vbucket 117 2016-04-07T03:43:46.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51600 Milliseconds Vbucket 117 2016-04-07T03:43:46.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51630 Milliseconds Vbucket 117 2016-04-07T03:43:46.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51660 Milliseconds Vbucket 117 2016-04-07T03:43:46.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51690 Milliseconds Vbucket 117 2016-04-07T03:43:46.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51720 Milliseconds Vbucket 117 2016-04-07T03:43:46.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51750 Milliseconds Vbucket 117 2016-04-07T03:43:46.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51780 Milliseconds Vbucket 117 2016-04-07T03:43:46.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51810 Milliseconds Vbucket 117 2016-04-07T03:43:46.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51840 Milliseconds Vbucket 117 2016-04-07T03:43:46.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51870 Milliseconds Vbucket 117 2016-04-07T03:43:46.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51900 Milliseconds Vbucket 117 2016-04-07T03:43:46.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51930 Milliseconds Vbucket 117 2016-04-07T03:43:46.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51960 Milliseconds Vbucket 117 2016-04-07T03:43:46.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51990 Milliseconds Vbucket 117 2016-04-07T03:43:46.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52020 Milliseconds Vbucket 117 2016-04-07T03:43:46.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52050 Milliseconds Vbucket 117 2016-04-07T03:43:46.778-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.308µs CommitTime 429.148329ms TotalFlushTime 3m54.273320339s TotalCommitTime 10.634369427s 2016-04-07T03:43:46.778-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 6623382, 6623382, 43 committed:true 2016-04-07T03:43:46.778-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 6623382, 6623382, 43 committed:true) SnapCreateDur 439.386301ms SnapOpenDur 42.837µs 2016-04-07T03:43:46.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52080 Milliseconds Vbucket 117 2016-04-07T03:43:46.825-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:46.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52110 Milliseconds Vbucket 117 2016-04-07T03:43:46.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52140 Milliseconds Vbucket 117 2016-04-07T03:43:46.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52170 Milliseconds Vbucket 117 2016-04-07T03:43:46.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52200 Milliseconds Vbucket 117 2016-04-07T03:43:46.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52230 Milliseconds Vbucket 117 2016-04-07T03:43:46.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52260 Milliseconds Vbucket 117 2016-04-07T03:43:47.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52290 Milliseconds Vbucket 117 2016-04-07T03:43:47.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52320 Milliseconds Vbucket 117 2016-04-07T03:43:47.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52350 Milliseconds Vbucket 117 2016-04-07T03:43:47.07-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6630000 QueuedCount 50000 2016-04-07T03:43:47.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52380 Milliseconds Vbucket 117 2016-04-07T03:43:47.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52410 Milliseconds Vbucket 117 2016-04-07T03:43:47.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52440 Milliseconds Vbucket 117 2016-04-07T03:43:47.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52470 Milliseconds Vbucket 117 2016-04-07T03:43:47.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52500 Milliseconds Vbucket 117 2016-04-07T03:43:47.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52530 Milliseconds Vbucket 117 2016-04-07T03:43:47.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52560 Milliseconds Vbucket 117 2016-04-07T03:43:47.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52590 Milliseconds Vbucket 117 2016-04-07T03:43:47.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52620 Milliseconds Vbucket 117 2016-04-07T03:43:47.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52650 Milliseconds Vbucket 117 2016-04-07T03:43:47.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52680 Milliseconds Vbucket 117 2016-04-07T03:43:47.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52710 Milliseconds Vbucket 117 2016-04-07T03:43:47.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52740 Milliseconds Vbucket 117 2016-04-07T03:43:47.467-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6640000 QueuedCount 50000 2016-04-07T03:43:47.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52770 Milliseconds Vbucket 117 2016-04-07T03:43:47.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52800 Milliseconds Vbucket 117 2016-04-07T03:43:47.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52830 Milliseconds Vbucket 117 2016-04-07T03:43:47.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52860 Milliseconds Vbucket 117 2016-04-07T03:43:47.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52890 Milliseconds Vbucket 117 2016-04-07T03:43:47.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52920 Milliseconds Vbucket 117 2016-04-07T03:43:47.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52950 Milliseconds Vbucket 117 2016-04-07T03:43:47.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52980 Milliseconds Vbucket 117 2016-04-07T03:43:47.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53010 Milliseconds Vbucket 117 2016-04-07T03:43:47.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53040 Milliseconds Vbucket 117 2016-04-07T03:43:47.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53070 Milliseconds Vbucket 117 2016-04-07T03:43:47.811-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6650000 QueuedCount 50000 2016-04-07T03:43:47.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53100 Milliseconds Vbucket 117 2016-04-07T03:43:47.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53130 Milliseconds Vbucket 117 2016-04-07T03:43:47.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53160 Milliseconds Vbucket 117 2016-04-07T03:43:47.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53190 Milliseconds Vbucket 117 2016-04-07T03:43:47.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53220 Milliseconds Vbucket 117 2016-04-07T03:43:47.96-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:47.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53250 Milliseconds Vbucket 117 2016-04-07T03:43:47.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53280 Milliseconds Vbucket 117 2016-04-07T03:43:48.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53310 Milliseconds Vbucket 117 2016-04-07T03:43:48.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53340 Milliseconds Vbucket 117 2016-04-07T03:43:48.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53370 Milliseconds Vbucket 117 2016-04-07T03:43:48.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53400 Milliseconds Vbucket 117 2016-04-07T03:43:48.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53430 Milliseconds Vbucket 117 2016-04-07T03:43:48.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53460 Milliseconds Vbucket 117 2016-04-07T03:43:48.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53490 Milliseconds Vbucket 117 2016-04-07T03:43:48.208-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6660000 QueuedCount 50000 2016-04-07T03:43:48.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53520 Milliseconds Vbucket 117 2016-04-07T03:43:48.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53550 Milliseconds Vbucket 117 2016-04-07T03:43:48.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53580 Milliseconds Vbucket 117 2016-04-07T03:43:48.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53610 Milliseconds Vbucket 117 2016-04-07T03:43:48.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53640 Milliseconds Vbucket 117 2016-04-07T03:43:48.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53670 Milliseconds Vbucket 117 2016-04-07T03:43:48.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53700 Milliseconds Vbucket 117 2016-04-07T03:43:48.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53730 Milliseconds Vbucket 117 2016-04-07T03:43:48.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53760 Milliseconds Vbucket 117 2016-04-07T03:43:48.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53790 Milliseconds Vbucket 117 2016-04-07T03:43:48.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53820 Milliseconds Vbucket 117 2016-04-07T03:43:48.552-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6670000 QueuedCount 49628 2016-04-07T03:43:48.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53850 Milliseconds Vbucket 117 2016-04-07T03:43:48.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53880 Milliseconds Vbucket 117 2016-04-07T03:43:48.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53910 Milliseconds Vbucket 117 2016-04-07T03:43:48.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53940 Milliseconds Vbucket 117 2016-04-07T03:43:48.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53970 Milliseconds Vbucket 117 2016-04-07T03:43:48.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54000 Milliseconds Vbucket 117 2016-04-07T03:43:48.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54030 Milliseconds Vbucket 117 2016-04-07T03:43:48.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54060 Milliseconds Vbucket 117 2016-04-07T03:43:48.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54090 Milliseconds Vbucket 117 2016-04-07T03:43:48.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54120 Milliseconds Vbucket 117 2016-04-07T03:43:48.86-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6680000 QueuedCount 39628 2016-04-07T03:43:48.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54150 Milliseconds Vbucket 117 2016-04-07T03:43:48.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54180 Milliseconds Vbucket 117 2016-04-07T03:43:48.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54210 Milliseconds Vbucket 117 2016-04-07T03:43:48.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54240 Milliseconds Vbucket 117 2016-04-07T03:43:48.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54270 Milliseconds Vbucket 117 2016-04-07T03:43:49.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54300 Milliseconds Vbucket 117 2016-04-07T03:43:49.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54330 Milliseconds Vbucket 117 2016-04-07T03:43:49.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54360 Milliseconds Vbucket 117 2016-04-07T03:43:49.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54390 Milliseconds Vbucket 117 2016-04-07T03:43:49.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54420 Milliseconds Vbucket 117 2016-04-07T03:43:49.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54450 Milliseconds Vbucket 117 2016-04-07T03:43:49.168-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6690000 QueuedCount 29628 2016-04-07T03:43:49.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54480 Milliseconds Vbucket 117 2016-04-07T03:43:49.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54510 Milliseconds Vbucket 117 2016-04-07T03:43:49.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54540 Milliseconds Vbucket 117 2016-04-07T03:43:49.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54570 Milliseconds Vbucket 117 2016-04-07T03:43:49.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54600 Milliseconds Vbucket 117 2016-04-07T03:43:49.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54630 Milliseconds Vbucket 117 2016-04-07T03:43:49.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54660 Milliseconds Vbucket 117 2016-04-07T03:43:49.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54690 Milliseconds Vbucket 117 2016-04-07T03:43:49.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54720 Milliseconds Vbucket 117 2016-04-07T03:43:49.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54750 Milliseconds Vbucket 117 2016-04-07T03:43:49.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54780 Milliseconds Vbucket 117 2016-04-07T03:43:49.505-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6700000 QueuedCount 19628 2016-04-07T03:43:49.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54810 Milliseconds Vbucket 117 2016-04-07T03:43:49.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54840 Milliseconds Vbucket 117 2016-04-07T03:43:49.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54870 Milliseconds Vbucket 117 2016-04-07T03:43:49.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54900 Milliseconds Vbucket 117 2016-04-07T03:43:49.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54930 Milliseconds Vbucket 117 2016-04-07T03:43:49.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54960 Milliseconds Vbucket 117 2016-04-07T03:43:49.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54990 Milliseconds Vbucket 117 2016-04-07T03:43:49.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55020 Milliseconds Vbucket 117 2016-04-07T03:43:49.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55050 Milliseconds Vbucket 117 2016-04-07T03:43:49.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55080 Milliseconds Vbucket 117 2016-04-07T03:43:49.824-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6710000 QueuedCount 9628 2016-04-07T03:43:49.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55110 Milliseconds Vbucket 117 2016-04-07T03:43:49.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55140 Milliseconds Vbucket 117 2016-04-07T03:43:49.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55170 Milliseconds Vbucket 117 2016-04-07T03:43:49.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55200 Milliseconds Vbucket 117 2016-04-07T03:43:49.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55230 Milliseconds Vbucket 117 2016-04-07T03:43:49.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55260 Milliseconds Vbucket 117 2016-04-07T03:43:50.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55290 Milliseconds Vbucket 117 2016-04-07T03:43:50.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55320 Milliseconds Vbucket 117 2016-04-07T03:43:50.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55350 Milliseconds Vbucket 117 2016-04-07T03:43:50.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55380 Milliseconds Vbucket 117 2016-04-07T03:43:50.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55410 Milliseconds Vbucket 117 2016-04-07T03:43:50.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55440 Milliseconds Vbucket 117 2016-04-07T03:43:50.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55470 Milliseconds Vbucket 117 2016-04-07T03:43:50.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55500 Milliseconds Vbucket 117 2016-04-07T03:43:50.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55530 Milliseconds Vbucket 117 2016-04-07T03:43:50.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55560 Milliseconds Vbucket 117 2016-04-07T03:43:50.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55590 Milliseconds Vbucket 117 2016-04-07T03:43:50.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55620 Milliseconds Vbucket 117 2016-04-07T03:43:50.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55650 Milliseconds Vbucket 117 2016-04-07T03:43:50.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55680 Milliseconds Vbucket 117 2016-04-07T03:43:50.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55710 Milliseconds Vbucket 117 2016-04-07T03:43:50.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55740 Milliseconds Vbucket 117 2016-04-07T03:43:50.467-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.564µs CommitTime 368.42884ms TotalFlushTime 3m57.439627746s TotalCommitTime 11.002798267s 2016-04-07T03:43:50.467-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 6719628, 6719628, 44 committed:true 2016-04-07T03:43:50.467-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 6719628, 6719628, 44 committed:true) SnapCreateDur 378.066249ms SnapOpenDur 56.554µs 2016-04-07T03:43:50.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55770 Milliseconds Vbucket 117 2016-04-07T03:43:50.493-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6720000 QueuedCount 11469 2016-04-07T03:43:50.503-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:50.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55800 Milliseconds Vbucket 117 2016-04-07T03:43:50.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55830 Milliseconds Vbucket 117 2016-04-07T03:43:50.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55860 Milliseconds Vbucket 117 2016-04-07T03:43:50.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55890 Milliseconds Vbucket 117 2016-04-07T03:43:50.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55920 Milliseconds Vbucket 117 2016-04-07T03:43:50.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55950 Milliseconds Vbucket 117 2016-04-07T03:43:50.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55980 Milliseconds Vbucket 117 2016-04-07T03:43:50.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56010 Milliseconds Vbucket 117 2016-04-07T03:43:50.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56040 Milliseconds Vbucket 117 2016-04-07T03:43:50.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56070 Milliseconds Vbucket 117 2016-04-07T03:43:50.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56100 Milliseconds Vbucket 117 2016-04-07T03:43:50.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56130 Milliseconds Vbucket 117 2016-04-07T03:43:50.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56160 Milliseconds Vbucket 117 2016-04-07T03:43:50.889-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6730000 QueuedCount 50000 2016-04-07T03:43:50.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56190 Milliseconds Vbucket 117 2016-04-07T03:43:50.909-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:50.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56220 Milliseconds Vbucket 117 2016-04-07T03:43:50.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56250 Milliseconds Vbucket 117 2016-04-07T03:43:50.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56280 Milliseconds Vbucket 117 2016-04-07T03:43:51.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56310 Milliseconds Vbucket 117 2016-04-07T03:43:51.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56340 Milliseconds Vbucket 117 2016-04-07T03:43:51.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56370 Milliseconds Vbucket 117 2016-04-07T03:43:51.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56400 Milliseconds Vbucket 117 2016-04-07T03:43:51.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56430 Milliseconds Vbucket 117 2016-04-07T03:43:51.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56460 Milliseconds Vbucket 117 2016-04-07T03:43:51.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56490 Milliseconds Vbucket 117 2016-04-07T03:43:51.223-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6740000 QueuedCount 50000 2016-04-07T03:43:51.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56520 Milliseconds Vbucket 117 2016-04-07T03:43:51.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56550 Milliseconds Vbucket 117 2016-04-07T03:43:51.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56580 Milliseconds Vbucket 117 2016-04-07T03:43:51.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56610 Milliseconds Vbucket 117 2016-04-07T03:43:51.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56640 Milliseconds Vbucket 117 2016-04-07T03:43:51.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56670 Milliseconds Vbucket 117 2016-04-07T03:43:51.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56700 Milliseconds Vbucket 117 2016-04-07T03:43:51.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56730 Milliseconds Vbucket 117 2016-04-07T03:43:51.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56760 Milliseconds Vbucket 117 2016-04-07T03:43:51.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56790 Milliseconds Vbucket 117 2016-04-07T03:43:51.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56820 Milliseconds Vbucket 117 2016-04-07T03:43:51.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56850 Milliseconds Vbucket 117 2016-04-07T03:43:51.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56880 Milliseconds Vbucket 117 2016-04-07T03:43:51.599-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6750000 QueuedCount 50000 2016-04-07T03:43:51.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56910 Milliseconds Vbucket 117 2016-04-07T03:43:51.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56940 Milliseconds Vbucket 117 2016-04-07T03:43:51.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56970 Milliseconds Vbucket 117 2016-04-07T03:43:51.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57000 Milliseconds Vbucket 117 2016-04-07T03:43:51.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57030 Milliseconds Vbucket 117 2016-04-07T03:43:51.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57060 Milliseconds Vbucket 117 2016-04-07T03:43:51.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57090 Milliseconds Vbucket 117 2016-04-07T03:43:51.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57120 Milliseconds Vbucket 117 2016-04-07T03:43:51.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57150 Milliseconds Vbucket 117 2016-04-07T03:43:51.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57180 Milliseconds Vbucket 117 2016-04-07T03:43:51.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57210 Milliseconds Vbucket 117 2016-04-07T03:43:51.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57240 Milliseconds Vbucket 117 2016-04-07T03:43:51.975-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6760000 QueuedCount 50000 2016-04-07T03:43:51.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57270 Milliseconds Vbucket 117 2016-04-07T03:43:52.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57300 Milliseconds Vbucket 117 2016-04-07T03:43:52.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57330 Milliseconds Vbucket 117 2016-04-07T03:43:52.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57360 Milliseconds Vbucket 117 2016-04-07T03:43:52.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57390 Milliseconds Vbucket 117 2016-04-07T03:43:52.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57420 Milliseconds Vbucket 117 2016-04-07T03:43:52.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57450 Milliseconds Vbucket 117 2016-04-07T03:43:52.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57480 Milliseconds Vbucket 117 2016-04-07T03:43:52.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57510 Milliseconds Vbucket 117 2016-04-07T03:43:52.246-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:52.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57540 Milliseconds Vbucket 117 2016-04-07T03:43:52.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57570 Milliseconds Vbucket 117 2016-04-07T03:43:52.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57600 Milliseconds Vbucket 117 2016-04-07T03:43:52.319-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6770000 QueuedCount 50000 2016-04-07T03:43:52.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57630 Milliseconds Vbucket 117 2016-04-07T03:43:52.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57660 Milliseconds Vbucket 117 2016-04-07T03:43:52.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57690 Milliseconds Vbucket 117 2016-04-07T03:43:52.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57720 Milliseconds Vbucket 117 2016-04-07T03:43:52.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57750 Milliseconds Vbucket 117 2016-04-07T03:43:52.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57780 Milliseconds Vbucket 117 2016-04-07T03:43:52.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57810 Milliseconds Vbucket 117 2016-04-07T03:43:52.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57840 Milliseconds Vbucket 117 2016-04-07T03:43:52.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57870 Milliseconds Vbucket 117 2016-04-07T03:43:52.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57900 Milliseconds Vbucket 117 2016-04-07T03:43:52.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57930 Milliseconds Vbucket 117 2016-04-07T03:43:52.667-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6780000 QueuedCount 50000 2016-04-07T03:43:52.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57960 Milliseconds Vbucket 117 2016-04-07T03:43:52.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57990 Milliseconds Vbucket 117 2016-04-07T03:43:52.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58020 Milliseconds Vbucket 117 2016-04-07T03:43:52.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58050 Milliseconds Vbucket 117 2016-04-07T03:43:52.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58080 Milliseconds Vbucket 117 2016-04-07T03:43:52.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58110 Milliseconds Vbucket 117 2016-04-07T03:43:52.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58140 Milliseconds Vbucket 117 2016-04-07T03:43:52.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58170 Milliseconds Vbucket 117 2016-04-07T03:43:52.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58200 Milliseconds Vbucket 117 2016-04-07T03:43:52.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58230 Milliseconds Vbucket 117 2016-04-07T03:43:52.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58260 Milliseconds Vbucket 117 2016-04-07T03:43:52.992-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6790000 QueuedCount 50000 2016-04-07T03:43:53.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58290 Milliseconds Vbucket 117 2016-04-07T03:43:53.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58320 Milliseconds Vbucket 117 2016-04-07T03:43:53.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58350 Milliseconds Vbucket 117 2016-04-07T03:43:53.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58380 Milliseconds Vbucket 117 2016-04-07T03:43:53.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58410 Milliseconds Vbucket 117 2016-04-07T03:43:53.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58440 Milliseconds Vbucket 117 2016-04-07T03:43:53.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58470 Milliseconds Vbucket 117 2016-04-07T03:43:53.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58500 Milliseconds Vbucket 117 2016-04-07T03:43:53.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58530 Milliseconds Vbucket 117 2016-04-07T03:43:53.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58560 Milliseconds Vbucket 117 2016-04-07T03:43:53.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58590 Milliseconds Vbucket 117 2016-04-07T03:43:53.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58620 Milliseconds Vbucket 117 2016-04-07T03:43:53.338-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6800000 QueuedCount 50000 2016-04-07T03:43:53.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58650 Milliseconds Vbucket 117 2016-04-07T03:43:53.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58680 Milliseconds Vbucket 117 2016-04-07T03:43:53.423-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:53.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58710 Milliseconds Vbucket 117 2016-04-07T03:43:53.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58740 Milliseconds Vbucket 117 2016-04-07T03:43:53.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58770 Milliseconds Vbucket 117 2016-04-07T03:43:53.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58800 Milliseconds Vbucket 117 2016-04-07T03:43:53.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58830 Milliseconds Vbucket 117 2016-04-07T03:43:53.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58860 Milliseconds Vbucket 117 2016-04-07T03:43:53.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58890 Milliseconds Vbucket 117 2016-04-07T03:43:53.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58920 Milliseconds Vbucket 117 2016-04-07T03:43:53.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58950 Milliseconds Vbucket 117 2016-04-07T03:43:53.688-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6810000 QueuedCount 50000 2016-04-07T03:43:53.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58980 Milliseconds Vbucket 117 2016-04-07T03:43:53.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59010 Milliseconds Vbucket 117 2016-04-07T03:43:53.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59040 Milliseconds Vbucket 117 2016-04-07T03:43:53.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59070 Milliseconds Vbucket 117 2016-04-07T03:43:53.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59100 Milliseconds Vbucket 117 2016-04-07T03:43:53.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59130 Milliseconds Vbucket 117 2016-04-07T03:43:53.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59160 Milliseconds Vbucket 117 2016-04-07T03:43:53.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59190 Milliseconds Vbucket 117 2016-04-07T03:43:53.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59220 Milliseconds Vbucket 117 2016-04-07T03:43:53.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59250 Milliseconds Vbucket 117 2016-04-07T03:43:53.981-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6820000 QueuedCount 50000 2016-04-07T03:43:53.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59280 Milliseconds Vbucket 117 2016-04-07T03:43:54.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59310 Milliseconds Vbucket 117 2016-04-07T03:43:54.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59340 Milliseconds Vbucket 117 2016-04-07T03:43:54.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59370 Milliseconds Vbucket 117 2016-04-07T03:43:54.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59400 Milliseconds Vbucket 117 2016-04-07T03:43:54.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59430 Milliseconds Vbucket 117 2016-04-07T03:43:54.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59460 Milliseconds Vbucket 117 2016-04-07T03:43:54.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59490 Milliseconds Vbucket 117 2016-04-07T03:43:54.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59520 Milliseconds Vbucket 117 2016-04-07T03:43:54.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59550 Milliseconds Vbucket 117 2016-04-07T03:43:54.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59580 Milliseconds Vbucket 117 2016-04-07T03:43:54.306-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6830000 QueuedCount 50000 2016-04-07T03:43:54.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59610 Milliseconds Vbucket 117 2016-04-07T03:43:54.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59640 Milliseconds Vbucket 117 2016-04-07T03:43:54.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59670 Milliseconds Vbucket 117 2016-04-07T03:43:54.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59700 Milliseconds Vbucket 117 2016-04-07T03:43:54.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59730 Milliseconds Vbucket 117 2016-04-07T03:43:54.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59760 Milliseconds Vbucket 117 2016-04-07T03:43:54.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59790 Milliseconds Vbucket 117 2016-04-07T03:43:54.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59820 Milliseconds Vbucket 117 2016-04-07T03:43:54.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59850 Milliseconds Vbucket 117 2016-04-07T03:43:54.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59880 Milliseconds Vbucket 117 2016-04-07T03:43:54.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59910 Milliseconds Vbucket 117 2016-04-07T03:43:54.635-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6840000 QueuedCount 49330 2016-04-07T03:43:54.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59940 Milliseconds Vbucket 117 2016-04-07T03:43:54.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59970 Milliseconds Vbucket 117 2016-04-07T03:43:54.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60000 Milliseconds Vbucket 117 2016-04-07T03:43:54.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60030 Milliseconds Vbucket 117 2016-04-07T03:43:54.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60060 Milliseconds Vbucket 117 2016-04-07T03:43:54.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60090 Milliseconds Vbucket 117 2016-04-07T03:43:54.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60120 Milliseconds Vbucket 117 2016-04-07T03:43:54.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60150 Milliseconds Vbucket 117 2016-04-07T03:43:54.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60180 Milliseconds Vbucket 117 2016-04-07T03:43:54.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60210 Milliseconds Vbucket 117 2016-04-07T03:43:54.937-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6850000 QueuedCount 39330 2016-04-07T03:43:54.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60240 Milliseconds Vbucket 117 2016-04-07T03:43:54.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60270 Milliseconds Vbucket 117 2016-04-07T03:43:55.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60300 Milliseconds Vbucket 117 2016-04-07T03:43:55.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60330 Milliseconds Vbucket 117 2016-04-07T03:43:55.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60360 Milliseconds Vbucket 117 2016-04-07T03:43:55.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60390 Milliseconds Vbucket 117 2016-04-07T03:43:55.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60420 Milliseconds Vbucket 117 2016-04-07T03:43:55.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60450 Milliseconds Vbucket 117 2016-04-07T03:43:55.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60480 Milliseconds Vbucket 117 2016-04-07T03:43:55.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60510 Milliseconds Vbucket 117 2016-04-07T03:43:55.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60540 Milliseconds Vbucket 117 2016-04-07T03:43:55.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60570 Milliseconds Vbucket 117 2016-04-07T03:43:55.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60600 Milliseconds Vbucket 117 2016-04-07T03:43:55.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60630 Milliseconds Vbucket 117 2016-04-07T03:43:55.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60660 Milliseconds Vbucket 117 2016-04-07T03:43:55.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60690 Milliseconds Vbucket 117 2016-04-07T03:43:55.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60720 Milliseconds Vbucket 117 2016-04-07T03:43:55.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60750 Milliseconds Vbucket 117 2016-04-07T03:43:55.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60780 Milliseconds Vbucket 117 2016-04-07T03:43:55.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60810 Milliseconds Vbucket 117 2016-04-07T03:43:55.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60840 Milliseconds Vbucket 117 2016-04-07T03:43:55.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60870 Milliseconds Vbucket 117 2016-04-07T03:43:55.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60900 Milliseconds Vbucket 117 2016-04-07T03:43:55.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60930 Milliseconds Vbucket 117 2016-04-07T03:43:55.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60960 Milliseconds Vbucket 117 2016-04-07T03:43:55.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60990 Milliseconds Vbucket 117 2016-04-07T03:43:55.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61020 Milliseconds Vbucket 117 2016-04-07T03:43:55.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61050 Milliseconds Vbucket 117 2016-04-07T03:43:55.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61080 Milliseconds Vbucket 117 2016-04-07T03:43:55.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61110 Milliseconds Vbucket 117 2016-04-07T03:43:55.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61140 Milliseconds Vbucket 117 2016-04-07T03:43:55.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61170 Milliseconds Vbucket 117 2016-04-07T03:43:55.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61200 Milliseconds Vbucket 117 2016-04-07T03:43:55.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61230 Milliseconds Vbucket 117 2016-04-07T03:43:55.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61260 Milliseconds Vbucket 117 2016-04-07T03:43:56.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61290 Milliseconds Vbucket 117 2016-04-07T03:43:56.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61320 Milliseconds Vbucket 117 2016-04-07T03:43:56.043-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6860000 QueuedCount 29330 2016-04-07T03:43:56.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61350 Milliseconds Vbucket 117 2016-04-07T03:43:56.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61380 Milliseconds Vbucket 117 2016-04-07T03:43:56.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61410 Milliseconds Vbucket 117 2016-04-07T03:43:56.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61440 Milliseconds Vbucket 117 2016-04-07T03:43:56.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61470 Milliseconds Vbucket 117 2016-04-07T03:43:56.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61500 Milliseconds Vbucket 117 2016-04-07T03:43:56.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61530 Milliseconds Vbucket 117 2016-04-07T03:43:56.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61560 Milliseconds Vbucket 117 2016-04-07T03:43:56.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61590 Milliseconds Vbucket 117 2016-04-07T03:43:56.331-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6870000 QueuedCount 19330 2016-04-07T03:43:56.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61620 Milliseconds Vbucket 117 2016-04-07T03:43:56.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61650 Milliseconds Vbucket 117 2016-04-07T03:43:56.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61680 Milliseconds Vbucket 117 2016-04-07T03:43:56.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61710 Milliseconds Vbucket 117 2016-04-07T03:43:56.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61740 Milliseconds Vbucket 117 2016-04-07T03:43:56.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61770 Milliseconds Vbucket 117 2016-04-07T03:43:56.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61800 Milliseconds Vbucket 117 2016-04-07T03:43:56.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61830 Milliseconds Vbucket 117 2016-04-07T03:43:56.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61860 Milliseconds Vbucket 117 2016-04-07T03:43:56.601-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6880000 QueuedCount 9330 2016-04-07T03:43:56.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61890 Milliseconds Vbucket 117 2016-04-07T03:43:56.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61920 Milliseconds Vbucket 117 2016-04-07T03:43:56.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61950 Milliseconds Vbucket 117 2016-04-07T03:43:56.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61980 Milliseconds Vbucket 117 2016-04-07T03:43:56.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62010 Milliseconds Vbucket 117 2016-04-07T03:43:56.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62040 Milliseconds Vbucket 117 2016-04-07T03:43:56.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62070 Milliseconds Vbucket 117 2016-04-07T03:43:56.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62100 Milliseconds Vbucket 117 2016-04-07T03:43:56.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62130 Milliseconds Vbucket 117 2016-04-07T03:43:56.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62160 Milliseconds Vbucket 117 2016-04-07T03:43:56.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62190 Milliseconds Vbucket 117 2016-04-07T03:43:56.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62220 Milliseconds Vbucket 117 2016-04-07T03:43:56.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62250 Milliseconds Vbucket 117 2016-04-07T03:43:56.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62280 Milliseconds Vbucket 117 2016-04-07T03:43:57.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62310 Milliseconds Vbucket 117 2016-04-07T03:43:57.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62340 Milliseconds Vbucket 117 2016-04-07T03:43:57.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62370 Milliseconds Vbucket 117 2016-04-07T03:43:57.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62400 Milliseconds Vbucket 117 2016-04-07T03:43:57.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62430 Milliseconds Vbucket 117 2016-04-07T03:43:57.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62460 Milliseconds Vbucket 117 2016-04-07T03:43:57.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62490 Milliseconds Vbucket 117 2016-04-07T03:43:57.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62520 Milliseconds Vbucket 117 2016-04-07T03:43:57.247-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.817µs CommitTime 388.035732ms TotalFlushTime 4m3.536732459s TotalCommitTime 11.390833999s 2016-04-07T03:43:57.247-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 6889330, 6889330, 45 committed:true 2016-04-07T03:43:57.247-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 6889330, 6889330, 45 committed:true) SnapCreateDur 398.276194ms SnapOpenDur 46.931µs 2016-04-07T03:43:57.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62550 Milliseconds Vbucket 117 2016-04-07T03:43:57.283-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6890000 QueuedCount 26610 2016-04-07T03:43:57.293-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:57.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62580 Milliseconds Vbucket 117 2016-04-07T03:43:57.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62610 Milliseconds Vbucket 117 2016-04-07T03:43:57.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62640 Milliseconds Vbucket 117 2016-04-07T03:43:57.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62670 Milliseconds Vbucket 117 2016-04-07T03:43:57.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62700 Milliseconds Vbucket 117 2016-04-07T03:43:57.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62730 Milliseconds Vbucket 117 2016-04-07T03:43:57.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62760 Milliseconds Vbucket 117 2016-04-07T03:43:57.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62790 Milliseconds Vbucket 117 2016-04-07T03:43:57.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62820 Milliseconds Vbucket 117 2016-04-07T03:43:57.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62850 Milliseconds Vbucket 117 2016-04-07T03:43:57.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62880 Milliseconds Vbucket 117 2016-04-07T03:43:57.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62910 Milliseconds Vbucket 117 2016-04-07T03:43:57.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62940 Milliseconds Vbucket 117 2016-04-07T03:43:57.68-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6900000 QueuedCount 50000 2016-04-07T03:43:57.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62970 Milliseconds Vbucket 117 2016-04-07T03:43:57.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63000 Milliseconds Vbucket 117 2016-04-07T03:43:57.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63030 Milliseconds Vbucket 117 2016-04-07T03:43:57.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63060 Milliseconds Vbucket 117 2016-04-07T03:43:57.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63090 Milliseconds Vbucket 117 2016-04-07T03:43:57.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63120 Milliseconds Vbucket 117 2016-04-07T03:43:57.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63150 Milliseconds Vbucket 117 2016-04-07T03:43:57.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63180 Milliseconds Vbucket 117 2016-04-07T03:43:57.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63210 Milliseconds Vbucket 117 2016-04-07T03:43:57.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63240 Milliseconds Vbucket 117 2016-04-07T03:43:57.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63270 Milliseconds Vbucket 117 2016-04-07T03:43:58.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63300 Milliseconds Vbucket 117 2016-04-07T03:43:58.037-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6910000 QueuedCount 50000 2016-04-07T03:43:58.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63330 Milliseconds Vbucket 117 2016-04-07T03:43:58.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63360 Milliseconds Vbucket 117 2016-04-07T03:43:58.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63390 Milliseconds Vbucket 117 2016-04-07T03:43:58.116-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:58.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63420 Milliseconds Vbucket 117 2016-04-07T03:43:58.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63450 Milliseconds Vbucket 117 2016-04-07T03:43:58.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63480 Milliseconds Vbucket 117 2016-04-07T03:43:58.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63510 Milliseconds Vbucket 117 2016-04-07T03:43:58.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63540 Milliseconds Vbucket 117 2016-04-07T03:43:58.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63570 Milliseconds Vbucket 117 2016-04-07T03:43:58.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63600 Milliseconds Vbucket 117 2016-04-07T03:43:58.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63630 Milliseconds Vbucket 117 2016-04-07T03:43:58.348-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6920000 QueuedCount 50000 2016-04-07T03:43:58.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63660 Milliseconds Vbucket 117 2016-04-07T03:43:58.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63690 Milliseconds Vbucket 117 2016-04-07T03:43:58.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63720 Milliseconds Vbucket 117 2016-04-07T03:43:58.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63750 Milliseconds Vbucket 117 2016-04-07T03:43:58.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63780 Milliseconds Vbucket 117 2016-04-07T03:43:58.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63810 Milliseconds Vbucket 117 2016-04-07T03:43:58.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63840 Milliseconds Vbucket 117 2016-04-07T03:43:58.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63870 Milliseconds Vbucket 117 2016-04-07T03:43:58.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63900 Milliseconds Vbucket 117 2016-04-07T03:43:58.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63930 Milliseconds Vbucket 117 2016-04-07T03:43:58.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63960 Milliseconds Vbucket 117 2016-04-07T03:43:58.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63990 Milliseconds Vbucket 117 2016-04-07T03:43:58.735-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6930000 QueuedCount 50000 2016-04-07T03:43:58.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64020 Milliseconds Vbucket 117 2016-04-07T03:43:58.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64050 Milliseconds Vbucket 117 2016-04-07T03:43:58.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64080 Milliseconds Vbucket 117 2016-04-07T03:43:58.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64110 Milliseconds Vbucket 117 2016-04-07T03:43:58.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64140 Milliseconds Vbucket 117 2016-04-07T03:43:58.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64170 Milliseconds Vbucket 117 2016-04-07T03:43:58.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64200 Milliseconds Vbucket 117 2016-04-07T03:43:58.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64230 Milliseconds Vbucket 117 2016-04-07T03:43:58.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64260 Milliseconds Vbucket 117 2016-04-07T03:43:59.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64290 Milliseconds Vbucket 117 2016-04-07T03:43:59.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64320 Milliseconds Vbucket 117 2016-04-07T03:43:59.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64350 Milliseconds Vbucket 117 2016-04-07T03:43:59.068-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6940000 QueuedCount 50000 2016-04-07T03:43:59.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64380 Milliseconds Vbucket 117 2016-04-07T03:43:59.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64410 Milliseconds Vbucket 117 2016-04-07T03:43:59.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64440 Milliseconds Vbucket 117 2016-04-07T03:43:59.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64470 Milliseconds Vbucket 117 2016-04-07T03:43:59.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64500 Milliseconds Vbucket 117 2016-04-07T03:43:59.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64530 Milliseconds Vbucket 117 2016-04-07T03:43:59.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64560 Milliseconds Vbucket 117 2016-04-07T03:43:59.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64590 Milliseconds Vbucket 117 2016-04-07T03:43:59.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64620 Milliseconds Vbucket 117 2016-04-07T03:43:59.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64650 Milliseconds Vbucket 117 2016-04-07T03:43:59.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64680 Milliseconds Vbucket 117 2016-04-07T03:43:59.4-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6950000 QueuedCount 50000 2016-04-07T03:43:59.401-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:43:59.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64710 Milliseconds Vbucket 117 2016-04-07T03:43:59.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64740 Milliseconds Vbucket 117 2016-04-07T03:43:59.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64770 Milliseconds Vbucket 117 2016-04-07T03:43:59.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64800 Milliseconds Vbucket 117 2016-04-07T03:43:59.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64830 Milliseconds Vbucket 117 2016-04-07T03:43:59.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64860 Milliseconds Vbucket 117 2016-04-07T03:43:59.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64890 Milliseconds Vbucket 117 2016-04-07T03:43:59.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64920 Milliseconds Vbucket 117 2016-04-07T03:43:59.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64950 Milliseconds Vbucket 117 2016-04-07T03:43:59.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64980 Milliseconds Vbucket 117 2016-04-07T03:43:59.71-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6960000 QueuedCount 50000 2016-04-07T03:43:59.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65010 Milliseconds Vbucket 117 2016-04-07T03:43:59.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65040 Milliseconds Vbucket 117 2016-04-07T03:43:59.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65070 Milliseconds Vbucket 117 2016-04-07T03:43:59.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65100 Milliseconds Vbucket 117 2016-04-07T03:43:59.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65130 Milliseconds Vbucket 117 2016-04-07T03:43:59.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65160 Milliseconds Vbucket 117 2016-04-07T03:43:59.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65190 Milliseconds Vbucket 117 2016-04-07T03:43:59.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65220 Milliseconds Vbucket 117 2016-04-07T03:43:59.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65250 Milliseconds Vbucket 117 2016-04-07T03:43:59.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65280 Milliseconds Vbucket 117 2016-04-07T03:44:00.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65310 Milliseconds Vbucket 117 2016-04-07T03:44:00.043-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6970000 QueuedCount 50000 2016-04-07T03:44:00.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65340 Milliseconds Vbucket 117 2016-04-07T03:44:00.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65370 Milliseconds Vbucket 117 2016-04-07T03:44:00.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65400 Milliseconds Vbucket 117 2016-04-07T03:44:00.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65430 Milliseconds Vbucket 117 2016-04-07T03:44:00.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65460 Milliseconds Vbucket 117 2016-04-07T03:44:00.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65490 Milliseconds Vbucket 117 2016-04-07T03:44:00.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65520 Milliseconds Vbucket 117 2016-04-07T03:44:00.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65550 Milliseconds Vbucket 117 2016-04-07T03:44:00.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65580 Milliseconds Vbucket 117 2016-04-07T03:44:00.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65610 Milliseconds Vbucket 117 2016-04-07T03:44:00.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65640 Milliseconds Vbucket 117 2016-04-07T03:44:00.378-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6980000 QueuedCount 50000 2016-04-07T03:44:00.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65670 Milliseconds Vbucket 117 2016-04-07T03:44:00.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65700 Milliseconds Vbucket 117 2016-04-07T03:44:00.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65730 Milliseconds Vbucket 117 2016-04-07T03:44:00.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65760 Milliseconds Vbucket 117 2016-04-07T03:44:00.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65790 Milliseconds Vbucket 117 2016-04-07T03:44:00.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65820 Milliseconds Vbucket 117 2016-04-07T03:44:00.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65850 Milliseconds Vbucket 117 2016-04-07T03:44:00.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65880 Milliseconds Vbucket 117 2016-04-07T03:44:00.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65910 Milliseconds Vbucket 117 2016-04-07T03:44:00.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65940 Milliseconds Vbucket 117 2016-04-07T03:44:00.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65970 Milliseconds Vbucket 117 2016-04-07T03:44:00.696-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 6990000 QueuedCount 47760 2016-04-07T03:44:00.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66000 Milliseconds Vbucket 117 2016-04-07T03:44:00.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66030 Milliseconds Vbucket 117 2016-04-07T03:44:00.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66060 Milliseconds Vbucket 117 2016-04-07T03:44:00.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66090 Milliseconds Vbucket 117 2016-04-07T03:44:00.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66120 Milliseconds Vbucket 117 2016-04-07T03:44:00.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66150 Milliseconds Vbucket 117 2016-04-07T03:44:00.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66180 Milliseconds Vbucket 117 2016-04-07T03:44:00.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66210 Milliseconds Vbucket 117 2016-04-07T03:44:00.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66240 Milliseconds Vbucket 117 2016-04-07T03:44:00.985-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7000000 QueuedCount 37760 2016-04-07T03:44:00.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66270 Milliseconds Vbucket 117 2016-04-07T03:44:01.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66300 Milliseconds Vbucket 117 2016-04-07T03:44:01.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66330 Milliseconds Vbucket 117 2016-04-07T03:44:01.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66360 Milliseconds Vbucket 117 2016-04-07T03:44:01.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66390 Milliseconds Vbucket 117 2016-04-07T03:44:01.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66420 Milliseconds Vbucket 117 2016-04-07T03:44:01.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66450 Milliseconds Vbucket 117 2016-04-07T03:44:01.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66480 Milliseconds Vbucket 117 2016-04-07T03:44:01.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66510 Milliseconds Vbucket 117 2016-04-07T03:44:01.254-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7010000 QueuedCount 27760 2016-04-07T03:44:01.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66540 Milliseconds Vbucket 117 2016-04-07T03:44:01.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66570 Milliseconds Vbucket 117 2016-04-07T03:44:01.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66600 Milliseconds Vbucket 117 2016-04-07T03:44:01.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66630 Milliseconds Vbucket 117 2016-04-07T03:44:01.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66660 Milliseconds Vbucket 117 2016-04-07T03:44:01.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66690 Milliseconds Vbucket 117 2016-04-07T03:44:01.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66720 Milliseconds Vbucket 117 2016-04-07T03:44:01.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66750 Milliseconds Vbucket 117 2016-04-07T03:44:01.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66780 Milliseconds Vbucket 117 2016-04-07T03:44:01.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66810 Milliseconds Vbucket 117 2016-04-07T03:44:01.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66840 Milliseconds Vbucket 117 2016-04-07T03:44:01.565-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7020000 QueuedCount 17760 2016-04-07T03:44:01.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66870 Milliseconds Vbucket 117 2016-04-07T03:44:01.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66900 Milliseconds Vbucket 117 2016-04-07T03:44:01.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66930 Milliseconds Vbucket 117 2016-04-07T03:44:01.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66960 Milliseconds Vbucket 117 2016-04-07T03:44:01.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66990 Milliseconds Vbucket 117 2016-04-07T03:44:01.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67020 Milliseconds Vbucket 117 2016-04-07T03:44:01.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67050 Milliseconds Vbucket 117 2016-04-07T03:44:01.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67080 Milliseconds Vbucket 117 2016-04-07T03:44:01.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67110 Milliseconds Vbucket 117 2016-04-07T03:44:01.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67140 Milliseconds Vbucket 117 2016-04-07T03:44:01.88-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7030000 QueuedCount 7760 2016-04-07T03:44:01.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67170 Milliseconds Vbucket 117 2016-04-07T03:44:01.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67200 Milliseconds Vbucket 117 2016-04-07T03:44:01.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67230 Milliseconds Vbucket 117 2016-04-07T03:44:01.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67260 Milliseconds Vbucket 117 2016-04-07T03:44:02.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67290 Milliseconds Vbucket 117 2016-04-07T03:44:02.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67320 Milliseconds Vbucket 117 2016-04-07T03:44:02.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67350 Milliseconds Vbucket 117 2016-04-07T03:44:02.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67380 Milliseconds Vbucket 117 2016-04-07T03:44:02.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67410 Milliseconds Vbucket 117 2016-04-07T03:44:02.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67440 Milliseconds Vbucket 117 2016-04-07T03:44:02.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67470 Milliseconds Vbucket 117 2016-04-07T03:44:02.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67500 Milliseconds Vbucket 117 2016-04-07T03:44:02.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67530 Milliseconds Vbucket 117 2016-04-07T03:44:02.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67560 Milliseconds Vbucket 117 2016-04-07T03:44:02.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67590 Milliseconds Vbucket 117 2016-04-07T03:44:02.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67620 Milliseconds Vbucket 117 2016-04-07T03:44:02.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67650 Milliseconds Vbucket 117 2016-04-07T03:44:02.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67680 Milliseconds Vbucket 117 2016-04-07T03:44:02.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67710 Milliseconds Vbucket 117 2016-04-07T03:44:02.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67740 Milliseconds Vbucket 117 2016-04-07T03:44:02.478-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.046µs CommitTime 330.295047ms TotalFlushTime 4m8.185638491s TotalCommitTime 11.721129046s 2016-04-07T03:44:02.478-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 7037760, 7037760, 46 committed:true 2016-04-07T03:44:02.478-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 7037760, 7037760, 46 committed:true) SnapCreateDur 340.926728ms SnapOpenDur 41.985µs 2016-04-07T03:44:02.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67770 Milliseconds Vbucket 117 2016-04-07T03:44:02.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67800 Milliseconds Vbucket 117 2016-04-07T03:44:02.521-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:02.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67830 Milliseconds Vbucket 117 2016-04-07T03:44:02.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67860 Milliseconds Vbucket 117 2016-04-07T03:44:02.596-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7040000 QueuedCount 50000 2016-04-07T03:44:02.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67890 Milliseconds Vbucket 117 2016-04-07T03:44:02.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67920 Milliseconds Vbucket 117 2016-04-07T03:44:02.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67950 Milliseconds Vbucket 117 2016-04-07T03:44:02.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67980 Milliseconds Vbucket 117 2016-04-07T03:44:02.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68010 Milliseconds Vbucket 117 2016-04-07T03:44:02.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68040 Milliseconds Vbucket 117 2016-04-07T03:44:02.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68070 Milliseconds Vbucket 117 2016-04-07T03:44:02.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68100 Milliseconds Vbucket 117 2016-04-07T03:44:02.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68130 Milliseconds Vbucket 117 2016-04-07T03:44:02.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68160 Milliseconds Vbucket 117 2016-04-07T03:44:02.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68190 Milliseconds Vbucket 117 2016-04-07T03:44:02.93-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7050000 QueuedCount 50000 2016-04-07T03:44:02.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68220 Milliseconds Vbucket 117 2016-04-07T03:44:02.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68250 Milliseconds Vbucket 117 2016-04-07T03:44:02.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68280 Milliseconds Vbucket 117 2016-04-07T03:44:03.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68310 Milliseconds Vbucket 117 2016-04-07T03:44:03.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68340 Milliseconds Vbucket 117 2016-04-07T03:44:03.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68370 Milliseconds Vbucket 117 2016-04-07T03:44:03.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68400 Milliseconds Vbucket 117 2016-04-07T03:44:03.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68430 Milliseconds Vbucket 117 2016-04-07T03:44:03.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68460 Milliseconds Vbucket 117 2016-04-07T03:44:03.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68490 Milliseconds Vbucket 117 2016-04-07T03:44:03.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68520 Milliseconds Vbucket 117 2016-04-07T03:44:03.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68550 Milliseconds Vbucket 117 2016-04-07T03:44:03.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68580 Milliseconds Vbucket 117 2016-04-07T03:44:03.317-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7060000 QueuedCount 50000 2016-04-07T03:44:03.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68610 Milliseconds Vbucket 117 2016-04-07T03:44:03.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68640 Milliseconds Vbucket 117 2016-04-07T03:44:03.378-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:03.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68670 Milliseconds Vbucket 117 2016-04-07T03:44:03.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68700 Milliseconds Vbucket 117 2016-04-07T03:44:03.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68730 Milliseconds Vbucket 117 2016-04-07T03:44:03.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68760 Milliseconds Vbucket 117 2016-04-07T03:44:03.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68790 Milliseconds Vbucket 117 2016-04-07T03:44:03.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68820 Milliseconds Vbucket 117 2016-04-07T03:44:03.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68850 Milliseconds Vbucket 117 2016-04-07T03:44:03.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68880 Milliseconds Vbucket 117 2016-04-07T03:44:03.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68910 Milliseconds Vbucket 117 2016-04-07T03:44:03.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68940 Milliseconds Vbucket 117 2016-04-07T03:44:03.674-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7070000 QueuedCount 50000 2016-04-07T03:44:03.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68970 Milliseconds Vbucket 117 2016-04-07T03:44:03.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69000 Milliseconds Vbucket 117 2016-04-07T03:44:03.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69030 Milliseconds Vbucket 117 2016-04-07T03:44:03.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69060 Milliseconds Vbucket 117 2016-04-07T03:44:03.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69090 Milliseconds Vbucket 117 2016-04-07T03:44:03.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69120 Milliseconds Vbucket 117 2016-04-07T03:44:03.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69150 Milliseconds Vbucket 117 2016-04-07T03:44:03.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69180 Milliseconds Vbucket 117 2016-04-07T03:44:03.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69210 Milliseconds Vbucket 117 2016-04-07T03:44:03.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69240 Milliseconds Vbucket 117 2016-04-07T03:44:03.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69270 Milliseconds Vbucket 117 2016-04-07T03:44:04.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69300 Milliseconds Vbucket 117 2016-04-07T03:44:04.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69330 Milliseconds Vbucket 117 2016-04-07T03:44:04.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7080000 QueuedCount 50000 2016-04-07T03:44:04.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69360 Milliseconds Vbucket 117 2016-04-07T03:44:04.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69390 Milliseconds Vbucket 117 2016-04-07T03:44:04.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69420 Milliseconds Vbucket 117 2016-04-07T03:44:04.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69450 Milliseconds Vbucket 117 2016-04-07T03:44:04.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69480 Milliseconds Vbucket 117 2016-04-07T03:44:04.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69510 Milliseconds Vbucket 117 2016-04-07T03:44:04.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69540 Milliseconds Vbucket 117 2016-04-07T03:44:04.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69570 Milliseconds Vbucket 117 2016-04-07T03:44:04.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69600 Milliseconds Vbucket 117 2016-04-07T03:44:04.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69630 Milliseconds Vbucket 117 2016-04-07T03:44:04.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69660 Milliseconds Vbucket 117 2016-04-07T03:44:04.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69690 Milliseconds Vbucket 117 2016-04-07T03:44:04.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69720 Milliseconds Vbucket 117 2016-04-07T03:44:04.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69750 Milliseconds Vbucket 117 2016-04-07T03:44:04.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69780 Milliseconds Vbucket 117 2016-04-07T03:44:04.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69810 Milliseconds Vbucket 117 2016-04-07T03:44:04.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69840 Milliseconds Vbucket 117 2016-04-07T03:44:04.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69870 Milliseconds Vbucket 117 2016-04-07T03:44:04.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69900 Milliseconds Vbucket 117 2016-04-07T03:44:04.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69930 Milliseconds Vbucket 117 2016-04-07T03:44:04.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69960 Milliseconds Vbucket 117 2016-04-07T03:44:04.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69990 Milliseconds Vbucket 117 2016-04-07T03:44:04.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70020 Milliseconds Vbucket 117 2016-04-07T03:44:04.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70050 Milliseconds Vbucket 117 2016-04-07T03:44:04.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70080 Milliseconds Vbucket 117 2016-04-07T03:44:04.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70110 Milliseconds Vbucket 117 2016-04-07T03:44:04.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70140 Milliseconds Vbucket 117 2016-04-07T03:44:04.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70170 Milliseconds Vbucket 117 2016-04-07T03:44:04.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70200 Milliseconds Vbucket 117 2016-04-07T03:44:04.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70230 Milliseconds Vbucket 117 2016-04-07T03:44:04.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70260 Milliseconds Vbucket 117 2016-04-07T03:44:05.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70290 Milliseconds Vbucket 117 2016-04-07T03:44:05.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70320 Milliseconds Vbucket 117 2016-04-07T03:44:05.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70350 Milliseconds Vbucket 117 2016-04-07T03:44:05.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70380 Milliseconds Vbucket 117 2016-04-07T03:44:05.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70410 Milliseconds Vbucket 117 2016-04-07T03:44:05.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70440 Milliseconds Vbucket 117 2016-04-07T03:44:05.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70470 Milliseconds Vbucket 117 2016-04-07T03:44:05.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70500 Milliseconds Vbucket 117 2016-04-07T03:44:05.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70530 Milliseconds Vbucket 117 2016-04-07T03:44:05.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70560 Milliseconds Vbucket 117 2016-04-07T03:44:05.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70590 Milliseconds Vbucket 117 2016-04-07T03:44:05.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70620 Milliseconds Vbucket 117 2016-04-07T03:44:05.362-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7090000 QueuedCount 50000 2016-04-07T03:44:05.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70650 Milliseconds Vbucket 117 2016-04-07T03:44:05.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70680 Milliseconds Vbucket 117 2016-04-07T03:44:05.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70710 Milliseconds Vbucket 117 2016-04-07T03:44:05.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70740 Milliseconds Vbucket 117 2016-04-07T03:44:05.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70770 Milliseconds Vbucket 117 2016-04-07T03:44:05.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70800 Milliseconds Vbucket 117 2016-04-07T03:44:05.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70830 Milliseconds Vbucket 117 2016-04-07T03:44:05.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70860 Milliseconds Vbucket 117 2016-04-07T03:44:05.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70890 Milliseconds Vbucket 117 2016-04-07T03:44:05.636-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:05.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70920 Milliseconds Vbucket 117 2016-04-07T03:44:05.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70950 Milliseconds Vbucket 117 2016-04-07T03:44:05.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70980 Milliseconds Vbucket 117 2016-04-07T03:44:05.716-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7100000 QueuedCount 50000 2016-04-07T03:44:05.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71010 Milliseconds Vbucket 117 2016-04-07T03:44:05.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71040 Milliseconds Vbucket 117 2016-04-07T03:44:05.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71070 Milliseconds Vbucket 117 2016-04-07T03:44:05.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71100 Milliseconds Vbucket 117 2016-04-07T03:44:05.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71130 Milliseconds Vbucket 117 2016-04-07T03:44:05.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71160 Milliseconds Vbucket 117 2016-04-07T03:44:05.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71190 Milliseconds Vbucket 117 2016-04-07T03:44:05.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71220 Milliseconds Vbucket 117 2016-04-07T03:44:05.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71250 Milliseconds Vbucket 117 2016-04-07T03:44:05.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71280 Milliseconds Vbucket 117 2016-04-07T03:44:06.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71310 Milliseconds Vbucket 117 2016-04-07T03:44:06.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71340 Milliseconds Vbucket 117 2016-04-07T03:44:06.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71370 Milliseconds Vbucket 117 2016-04-07T03:44:06.111-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7110000 QueuedCount 50000 2016-04-07T03:44:06.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71400 Milliseconds Vbucket 117 2016-04-07T03:44:06.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71430 Milliseconds Vbucket 117 2016-04-07T03:44:06.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71460 Milliseconds Vbucket 117 2016-04-07T03:44:06.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71490 Milliseconds Vbucket 117 2016-04-07T03:44:06.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71520 Milliseconds Vbucket 117 2016-04-07T03:44:06.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71550 Milliseconds Vbucket 117 2016-04-07T03:44:06.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71580 Milliseconds Vbucket 117 2016-04-07T03:44:06.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71610 Milliseconds Vbucket 117 2016-04-07T03:44:06.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71640 Milliseconds Vbucket 117 2016-04-07T03:44:06.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71670 Milliseconds Vbucket 117 2016-04-07T03:44:06.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71700 Milliseconds Vbucket 117 2016-04-07T03:44:06.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71730 Milliseconds Vbucket 117 2016-04-07T03:44:06.466-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7120000 QueuedCount 50000 2016-04-07T03:44:06.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71760 Milliseconds Vbucket 117 2016-04-07T03:44:06.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71790 Milliseconds Vbucket 117 2016-04-07T03:44:06.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71820 Milliseconds Vbucket 117 2016-04-07T03:44:06.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71850 Milliseconds Vbucket 117 2016-04-07T03:44:06.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71880 Milliseconds Vbucket 117 2016-04-07T03:44:06.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71910 Milliseconds Vbucket 117 2016-04-07T03:44:06.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71940 Milliseconds Vbucket 117 2016-04-07T03:44:06.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71970 Milliseconds Vbucket 117 2016-04-07T03:44:06.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72000 Milliseconds Vbucket 117 2016-04-07T03:44:06.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72030 Milliseconds Vbucket 117 2016-04-07T03:44:06.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72060 Milliseconds Vbucket 117 2016-04-07T03:44:06.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72090 Milliseconds Vbucket 117 2016-04-07T03:44:06.813-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7130000 QueuedCount 50000 2016-04-07T03:44:06.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72120 Milliseconds Vbucket 117 2016-04-07T03:44:06.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72150 Milliseconds Vbucket 117 2016-04-07T03:44:06.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72180 Milliseconds Vbucket 117 2016-04-07T03:44:06.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72210 Milliseconds Vbucket 117 2016-04-07T03:44:06.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72240 Milliseconds Vbucket 117 2016-04-07T03:44:06.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72270 Milliseconds Vbucket 117 2016-04-07T03:44:07.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72300 Milliseconds Vbucket 117 2016-04-07T03:44:07.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72330 Milliseconds Vbucket 117 2016-04-07T03:44:07.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72360 Milliseconds Vbucket 117 2016-04-07T03:44:07.098-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7140000 QueuedCount 45063 2016-04-07T03:44:07.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72390 Milliseconds Vbucket 117 2016-04-07T03:44:07.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72420 Milliseconds Vbucket 117 2016-04-07T03:44:07.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72450 Milliseconds Vbucket 117 2016-04-07T03:44:07.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72480 Milliseconds Vbucket 117 2016-04-07T03:44:07.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72510 Milliseconds Vbucket 117 2016-04-07T03:44:07.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72540 Milliseconds Vbucket 117 2016-04-07T03:44:07.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72570 Milliseconds Vbucket 117 2016-04-07T03:44:07.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72600 Milliseconds Vbucket 117 2016-04-07T03:44:07.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72630 Milliseconds Vbucket 117 2016-04-07T03:44:07.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72660 Milliseconds Vbucket 117 2016-04-07T03:44:07.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72690 Milliseconds Vbucket 117 2016-04-07T03:44:07.414-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7150000 QueuedCount 35063 2016-04-07T03:44:07.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72720 Milliseconds Vbucket 117 2016-04-07T03:44:07.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72750 Milliseconds Vbucket 117 2016-04-07T03:44:07.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72780 Milliseconds Vbucket 117 2016-04-07T03:44:07.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72810 Milliseconds Vbucket 117 2016-04-07T03:44:07.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72840 Milliseconds Vbucket 117 2016-04-07T03:44:07.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72870 Milliseconds Vbucket 117 2016-04-07T03:44:07.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72900 Milliseconds Vbucket 117 2016-04-07T03:44:07.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72930 Milliseconds Vbucket 117 2016-04-07T03:44:07.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72960 Milliseconds Vbucket 117 2016-04-07T03:44:07.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72990 Milliseconds Vbucket 117 2016-04-07T03:44:07.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73020 Milliseconds Vbucket 117 2016-04-07T03:44:07.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73050 Milliseconds Vbucket 117 2016-04-07T03:44:07.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73080 Milliseconds Vbucket 117 2016-04-07T03:44:07.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73110 Milliseconds Vbucket 117 2016-04-07T03:44:07.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73140 Milliseconds Vbucket 117 2016-04-07T03:44:07.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73170 Milliseconds Vbucket 117 2016-04-07T03:44:07.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73200 Milliseconds Vbucket 117 2016-04-07T03:44:07.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73230 Milliseconds Vbucket 117 2016-04-07T03:44:07.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73260 Milliseconds Vbucket 117 2016-04-07T03:44:08.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73290 Milliseconds Vbucket 117 2016-04-07T03:44:08.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73320 Milliseconds Vbucket 117 2016-04-07T03:44:08.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73350 Milliseconds Vbucket 117 2016-04-07T03:44:08.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73380 Milliseconds Vbucket 117 2016-04-07T03:44:08.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73410 Milliseconds Vbucket 117 2016-04-07T03:44:08.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73440 Milliseconds Vbucket 117 2016-04-07T03:44:08.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73470 Milliseconds Vbucket 117 2016-04-07T03:44:08.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73500 Milliseconds Vbucket 117 2016-04-07T03:44:08.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73530 Milliseconds Vbucket 117 2016-04-07T03:44:08.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73560 Milliseconds Vbucket 117 2016-04-07T03:44:08.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73590 Milliseconds Vbucket 117 2016-04-07T03:44:08.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73620 Milliseconds Vbucket 117 2016-04-07T03:44:08.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73650 Milliseconds Vbucket 117 2016-04-07T03:44:08.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73680 Milliseconds Vbucket 117 2016-04-07T03:44:08.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73710 Milliseconds Vbucket 117 2016-04-07T03:44:08.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73740 Milliseconds Vbucket 117 2016-04-07T03:44:08.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73770 Milliseconds Vbucket 117 2016-04-07T03:44:08.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73800 Milliseconds Vbucket 117 2016-04-07T03:44:08.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73830 Milliseconds Vbucket 117 2016-04-07T03:44:08.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73860 Milliseconds Vbucket 117 2016-04-07T03:44:08.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73890 Milliseconds Vbucket 117 2016-04-07T03:44:08.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73920 Milliseconds Vbucket 117 2016-04-07T03:44:08.638-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7160000 QueuedCount 25063 2016-04-07T03:44:08.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73950 Milliseconds Vbucket 117 2016-04-07T03:44:08.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73980 Milliseconds Vbucket 117 2016-04-07T03:44:08.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74010 Milliseconds Vbucket 117 2016-04-07T03:44:08.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74040 Milliseconds Vbucket 117 2016-04-07T03:44:08.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74070 Milliseconds Vbucket 117 2016-04-07T03:44:08.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74100 Milliseconds Vbucket 117 2016-04-07T03:44:08.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74130 Milliseconds Vbucket 117 2016-04-07T03:44:08.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74160 Milliseconds Vbucket 117 2016-04-07T03:44:08.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74190 Milliseconds Vbucket 117 2016-04-07T03:44:08.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74220 Milliseconds Vbucket 117 2016-04-07T03:44:08.949-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7170000 QueuedCount 15063 2016-04-07T03:44:08.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74250 Milliseconds Vbucket 117 2016-04-07T03:44:08.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74280 Milliseconds Vbucket 117 2016-04-07T03:44:09.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74310 Milliseconds Vbucket 117 2016-04-07T03:44:09.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74340 Milliseconds Vbucket 117 2016-04-07T03:44:09.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74370 Milliseconds Vbucket 117 2016-04-07T03:44:09.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74400 Milliseconds Vbucket 117 2016-04-07T03:44:09.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74430 Milliseconds Vbucket 117 2016-04-07T03:44:09.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74460 Milliseconds Vbucket 117 2016-04-07T03:44:09.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74490 Milliseconds Vbucket 117 2016-04-07T03:44:09.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74520 Milliseconds Vbucket 117 2016-04-07T03:44:09.259-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7180000 QueuedCount 5063 2016-04-07T03:44:09.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74550 Milliseconds Vbucket 117 2016-04-07T03:44:09.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74580 Milliseconds Vbucket 117 2016-04-07T03:44:09.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74610 Milliseconds Vbucket 117 2016-04-07T03:44:09.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74640 Milliseconds Vbucket 117 2016-04-07T03:44:09.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74670 Milliseconds Vbucket 117 2016-04-07T03:44:09.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74700 Milliseconds Vbucket 117 2016-04-07T03:44:09.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74730 Milliseconds Vbucket 117 2016-04-07T03:44:09.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74760 Milliseconds Vbucket 117 2016-04-07T03:44:09.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74790 Milliseconds Vbucket 117 2016-04-07T03:44:09.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74820 Milliseconds Vbucket 117 2016-04-07T03:44:09.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74850 Milliseconds Vbucket 117 2016-04-07T03:44:09.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74880 Milliseconds Vbucket 117 2016-04-07T03:44:09.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74910 Milliseconds Vbucket 117 2016-04-07T03:44:09.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74940 Milliseconds Vbucket 117 2016-04-07T03:44:09.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74970 Milliseconds Vbucket 117 2016-04-07T03:44:09.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75000 Milliseconds Vbucket 117 2016-04-07T03:44:09.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75030 Milliseconds Vbucket 117 2016-04-07T03:44:09.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75060 Milliseconds Vbucket 117 2016-04-07T03:44:09.791-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.931µs CommitTime 379.839275ms TotalFlushTime 4m14.849039814s TotalCommitTime 12.100968321s 2016-04-07T03:44:09.791-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 7185063, 7185063, 47 committed:true 2016-04-07T03:44:09.791-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 7185063, 7185063, 47 committed:true) SnapCreateDur 389.620362ms SnapOpenDur 54.987µs 2016-04-07T03:44:09.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75090 Milliseconds Vbucket 117 2016-04-07T03:44:09.836-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:09.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75120 Milliseconds Vbucket 117 2016-04-07T03:44:09.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75150 Milliseconds Vbucket 117 2016-04-07T03:44:09.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75180 Milliseconds Vbucket 117 2016-04-07T03:44:09.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75210 Milliseconds Vbucket 117 2016-04-07T03:44:09.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75240 Milliseconds Vbucket 117 2016-04-07T03:44:09.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75270 Milliseconds Vbucket 117 2016-04-07T03:44:09.996-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7190000 QueuedCount 50000 2016-04-07T03:44:10.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75300 Milliseconds Vbucket 117 2016-04-07T03:44:10.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75330 Milliseconds Vbucket 117 2016-04-07T03:44:10.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75360 Milliseconds Vbucket 117 2016-04-07T03:44:10.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75390 Milliseconds Vbucket 117 2016-04-07T03:44:10.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75420 Milliseconds Vbucket 117 2016-04-07T03:44:10.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75450 Milliseconds Vbucket 117 2016-04-07T03:44:10.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75480 Milliseconds Vbucket 117 2016-04-07T03:44:10.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75510 Milliseconds Vbucket 117 2016-04-07T03:44:10.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75540 Milliseconds Vbucket 117 2016-04-07T03:44:10.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75570 Milliseconds Vbucket 117 2016-04-07T03:44:10.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75600 Milliseconds Vbucket 117 2016-04-07T03:44:10.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75630 Milliseconds Vbucket 117 2016-04-07T03:44:10.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75660 Milliseconds Vbucket 117 2016-04-07T03:44:10.393-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7200000 QueuedCount 50000 2016-04-07T03:44:10.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75690 Milliseconds Vbucket 117 2016-04-07T03:44:10.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75720 Milliseconds Vbucket 117 2016-04-07T03:44:10.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75750 Milliseconds Vbucket 117 2016-04-07T03:44:10.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75780 Milliseconds Vbucket 117 2016-04-07T03:44:10.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75810 Milliseconds Vbucket 117 2016-04-07T03:44:10.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75840 Milliseconds Vbucket 117 2016-04-07T03:44:10.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75870 Milliseconds Vbucket 117 2016-04-07T03:44:10.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75900 Milliseconds Vbucket 117 2016-04-07T03:44:10.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75930 Milliseconds Vbucket 117 2016-04-07T03:44:10.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75960 Milliseconds Vbucket 117 2016-04-07T03:44:10.696-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:10.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75990 Milliseconds Vbucket 117 2016-04-07T03:44:10.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76020 Milliseconds Vbucket 117 2016-04-07T03:44:10.755-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7210000 QueuedCount 50000 2016-04-07T03:44:10.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76050 Milliseconds Vbucket 117 2016-04-07T03:44:10.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76080 Milliseconds Vbucket 117 2016-04-07T03:44:10.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76110 Milliseconds Vbucket 117 2016-04-07T03:44:10.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76140 Milliseconds Vbucket 117 2016-04-07T03:44:10.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76170 Milliseconds Vbucket 117 2016-04-07T03:44:10.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76200 Milliseconds Vbucket 117 2016-04-07T03:44:10.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76230 Milliseconds Vbucket 117 2016-04-07T03:44:10.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76260 Milliseconds Vbucket 117 2016-04-07T03:44:11.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76290 Milliseconds Vbucket 117 2016-04-07T03:44:11.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76320 Milliseconds Vbucket 117 2016-04-07T03:44:11.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76350 Milliseconds Vbucket 117 2016-04-07T03:44:11.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76380 Milliseconds Vbucket 117 2016-04-07T03:44:11.108-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7220000 QueuedCount 50000 2016-04-07T03:44:11.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76410 Milliseconds Vbucket 117 2016-04-07T03:44:11.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76440 Milliseconds Vbucket 117 2016-04-07T03:44:11.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76470 Milliseconds Vbucket 117 2016-04-07T03:44:11.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76500 Milliseconds Vbucket 117 2016-04-07T03:44:11.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76530 Milliseconds Vbucket 117 2016-04-07T03:44:11.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76560 Milliseconds Vbucket 117 2016-04-07T03:44:11.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76590 Milliseconds Vbucket 117 2016-04-07T03:44:11.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76620 Milliseconds Vbucket 117 2016-04-07T03:44:11.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76650 Milliseconds Vbucket 117 2016-04-07T03:44:11.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76680 Milliseconds Vbucket 117 2016-04-07T03:44:11.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76710 Milliseconds Vbucket 117 2016-04-07T03:44:11.451-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7230000 QueuedCount 50000 2016-04-07T03:44:11.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76740 Milliseconds Vbucket 117 2016-04-07T03:44:11.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76770 Milliseconds Vbucket 117 2016-04-07T03:44:11.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76800 Milliseconds Vbucket 117 2016-04-07T03:44:11.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76830 Milliseconds Vbucket 117 2016-04-07T03:44:11.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76860 Milliseconds Vbucket 117 2016-04-07T03:44:11.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76890 Milliseconds Vbucket 117 2016-04-07T03:44:11.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76920 Milliseconds Vbucket 117 2016-04-07T03:44:11.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76950 Milliseconds Vbucket 117 2016-04-07T03:44:11.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76980 Milliseconds Vbucket 117 2016-04-07T03:44:11.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77010 Milliseconds Vbucket 117 2016-04-07T03:44:11.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77040 Milliseconds Vbucket 117 2016-04-07T03:44:11.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77070 Milliseconds Vbucket 117 2016-04-07T03:44:11.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77100 Milliseconds Vbucket 117 2016-04-07T03:44:11.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77130 Milliseconds Vbucket 117 2016-04-07T03:44:11.849-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7240000 QueuedCount 50000 2016-04-07T03:44:11.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77160 Milliseconds Vbucket 117 2016-04-07T03:44:11.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77190 Milliseconds Vbucket 117 2016-04-07T03:44:11.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77220 Milliseconds Vbucket 117 2016-04-07T03:44:11.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77250 Milliseconds Vbucket 117 2016-04-07T03:44:11.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77280 Milliseconds Vbucket 117 2016-04-07T03:44:12.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77310 Milliseconds Vbucket 117 2016-04-07T03:44:12.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77340 Milliseconds Vbucket 117 2016-04-07T03:44:12.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77370 Milliseconds Vbucket 117 2016-04-07T03:44:12.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77400 Milliseconds Vbucket 117 2016-04-07T03:44:12.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77430 Milliseconds Vbucket 117 2016-04-07T03:44:12.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77460 Milliseconds Vbucket 117 2016-04-07T03:44:12.195-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7250000 QueuedCount 46021 2016-04-07T03:44:12.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77490 Milliseconds Vbucket 117 2016-04-07T03:44:12.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77520 Milliseconds Vbucket 117 2016-04-07T03:44:12.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77550 Milliseconds Vbucket 117 2016-04-07T03:44:12.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77580 Milliseconds Vbucket 117 2016-04-07T03:44:12.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77610 Milliseconds Vbucket 117 2016-04-07T03:44:12.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77640 Milliseconds Vbucket 117 2016-04-07T03:44:12.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77670 Milliseconds Vbucket 117 2016-04-07T03:44:12.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77700 Milliseconds Vbucket 117 2016-04-07T03:44:12.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77730 Milliseconds Vbucket 117 2016-04-07T03:44:12.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77760 Milliseconds Vbucket 117 2016-04-07T03:44:12.498-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7260000 QueuedCount 36021 2016-04-07T03:44:12.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77790 Milliseconds Vbucket 117 2016-04-07T03:44:12.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77820 Milliseconds Vbucket 117 2016-04-07T03:44:12.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77850 Milliseconds Vbucket 117 2016-04-07T03:44:12.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77880 Milliseconds Vbucket 117 2016-04-07T03:44:12.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77910 Milliseconds Vbucket 117 2016-04-07T03:44:12.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77940 Milliseconds Vbucket 117 2016-04-07T03:44:12.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77970 Milliseconds Vbucket 117 2016-04-07T03:44:12.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78000 Milliseconds Vbucket 117 2016-04-07T03:44:12.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78030 Milliseconds Vbucket 117 2016-04-07T03:44:12.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78060 Milliseconds Vbucket 117 2016-04-07T03:44:12.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78090 Milliseconds Vbucket 117 2016-04-07T03:44:12.814-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7270000 QueuedCount 26021 2016-04-07T03:44:12.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78120 Milliseconds Vbucket 117 2016-04-07T03:44:12.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78150 Milliseconds Vbucket 117 2016-04-07T03:44:12.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78180 Milliseconds Vbucket 117 2016-04-07T03:44:12.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78210 Milliseconds Vbucket 117 2016-04-07T03:44:12.941-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":6991574,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":6404194156,"bucket-1:myindex:avg_ts_items_count":145498,"bucket-1:myindex:build_progress":36,"bucket-1:myindex:data_size":2318480439,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":8845803520,"bucket-1:myindex:flush_queue_size":21746,"bucket-1:myindex:frag_percent":73,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":763680540,"bucket-1:myindex:items_count":7185063,"bucket-1:myindex:num_commits":47,"bucket-1:myindex:num_compactions":2,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":7259123,"bucket-1:myindex:num_docs_pending":5712403,"bucket-1:myindex:num_docs_queued":7028474,"bucket-1:myindex:num_flush_queued":7296021,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":47,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":7313281689,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"47 12100968321 3620897018570209511","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"7274277 46168110558 610504067863468924","bucket-1:myindex:timings/storage_handle_open":"47 1128483 27944727037","bucket-1:myindex:timings/storage_info":"189 1718285 40609157729","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"48 52556016 724053458945098","bucket-1:myindex:timings/storage_meta_set":"47 345236880 107821639752203544","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"14548554 207012234966 7087455185019347162","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":14287595,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":263,"memory_quota":31457280000,"memory_used":4277411840,"needs_restart":false,"num_connections":0,"timings/stats_response":"1630 156904624 24709012288078"} 2016-04-07T03:44:12.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78240 Milliseconds Vbucket 117 2016-04-07T03:44:12.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78270 Milliseconds Vbucket 117 2016-04-07T03:44:13.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78300 Milliseconds Vbucket 117 2016-04-07T03:44:13.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78330 Milliseconds Vbucket 117 2016-04-07T03:44:13.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78360 Milliseconds Vbucket 117 2016-04-07T03:44:13.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78390 Milliseconds Vbucket 117 2016-04-07T03:44:13.123-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7280000 QueuedCount 16021 2016-04-07T03:44:13.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78420 Milliseconds Vbucket 117 2016-04-07T03:44:13.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78450 Milliseconds Vbucket 117 2016-04-07T03:44:13.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78480 Milliseconds Vbucket 117 2016-04-07T03:44:13.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78510 Milliseconds Vbucket 117 2016-04-07T03:44:13.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78540 Milliseconds Vbucket 117 2016-04-07T03:44:13.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78570 Milliseconds Vbucket 117 2016-04-07T03:44:13.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78600 Milliseconds Vbucket 117 2016-04-07T03:44:13.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78630 Milliseconds Vbucket 117 2016-04-07T03:44:13.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78660 Milliseconds Vbucket 117 2016-04-07T03:44:13.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78690 Milliseconds Vbucket 117 2016-04-07T03:44:13.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78720 Milliseconds Vbucket 117 2016-04-07T03:44:13.443-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7290000 QueuedCount 6021 2016-04-07T03:44:13.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78750 Milliseconds Vbucket 117 2016-04-07T03:44:13.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78780 Milliseconds Vbucket 117 2016-04-07T03:44:13.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78810 Milliseconds Vbucket 117 2016-04-07T03:44:13.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78840 Milliseconds Vbucket 117 2016-04-07T03:44:13.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78870 Milliseconds Vbucket 117 2016-04-07T03:44:13.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78900 Milliseconds Vbucket 117 2016-04-07T03:44:13.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78930 Milliseconds Vbucket 117 2016-04-07T03:44:13.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78960 Milliseconds Vbucket 117 2016-04-07T03:44:13.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78990 Milliseconds Vbucket 117 2016-04-07T03:44:13.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79020 Milliseconds Vbucket 117 2016-04-07T03:44:13.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79050 Milliseconds Vbucket 117 2016-04-07T03:44:13.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79080 Milliseconds Vbucket 117 2016-04-07T03:44:13.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79110 Milliseconds Vbucket 117 2016-04-07T03:44:13.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79140 Milliseconds Vbucket 117 2016-04-07T03:44:13.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79170 Milliseconds Vbucket 117 2016-04-07T03:44:13.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79200 Milliseconds Vbucket 117 2016-04-07T03:44:13.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79230 Milliseconds Vbucket 117 2016-04-07T03:44:13.956-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.615µs CommitTime 310.918904ms TotalFlushTime 4m18.50875533s TotalCommitTime 12.411887225s 2016-04-07T03:44:13.956-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 7296021, 7296021, 48 committed:true 2016-04-07T03:44:13.956-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 7296021, 7296021, 48 committed:true) SnapCreateDur 320.851865ms SnapOpenDur 55.11µs 2016-04-07T03:44:13.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79260 Milliseconds Vbucket 117 2016-04-07T03:44:14.001-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:14.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79290 Milliseconds Vbucket 117 2016-04-07T03:44:14.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79320 Milliseconds Vbucket 117 2016-04-07T03:44:14.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79350 Milliseconds Vbucket 117 2016-04-07T03:44:14.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79380 Milliseconds Vbucket 117 2016-04-07T03:44:14.1-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7300000 QueuedCount 50000 2016-04-07T03:44:14.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79410 Milliseconds Vbucket 117 2016-04-07T03:44:14.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79440 Milliseconds Vbucket 117 2016-04-07T03:44:14.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79470 Milliseconds Vbucket 117 2016-04-07T03:44:14.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79500 Milliseconds Vbucket 117 2016-04-07T03:44:14.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79530 Milliseconds Vbucket 117 2016-04-07T03:44:14.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79560 Milliseconds Vbucket 117 2016-04-07T03:44:14.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79590 Milliseconds Vbucket 117 2016-04-07T03:44:14.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79620 Milliseconds Vbucket 117 2016-04-07T03:44:14.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79650 Milliseconds Vbucket 117 2016-04-07T03:44:14.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79680 Milliseconds Vbucket 117 2016-04-07T03:44:14.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79710 Milliseconds Vbucket 117 2016-04-07T03:44:14.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79740 Milliseconds Vbucket 117 2016-04-07T03:44:14.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79770 Milliseconds Vbucket 117 2016-04-07T03:44:14.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79800 Milliseconds Vbucket 117 2016-04-07T03:44:14.521-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7310000 QueuedCount 50000 2016-04-07T03:44:14.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79830 Milliseconds Vbucket 117 2016-04-07T03:44:14.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79860 Milliseconds Vbucket 117 2016-04-07T03:44:14.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79890 Milliseconds Vbucket 117 2016-04-07T03:44:14.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79920 Milliseconds Vbucket 117 2016-04-07T03:44:14.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79950 Milliseconds Vbucket 117 2016-04-07T03:44:14.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79980 Milliseconds Vbucket 117 2016-04-07T03:44:14.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80010 Milliseconds Vbucket 117 2016-04-07T03:44:14.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80040 Milliseconds Vbucket 117 2016-04-07T03:44:14.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80070 Milliseconds Vbucket 117 2016-04-07T03:44:14.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80100 Milliseconds Vbucket 117 2016-04-07T03:44:14.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80130 Milliseconds Vbucket 117 2016-04-07T03:44:14.866-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:14.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80160 Milliseconds Vbucket 117 2016-04-07T03:44:14.878-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7320000 QueuedCount 48852 2016-04-07T03:44:14.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80190 Milliseconds Vbucket 117 2016-04-07T03:44:14.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80220 Milliseconds Vbucket 117 2016-04-07T03:44:14.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80250 Milliseconds Vbucket 117 2016-04-07T03:44:14.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80280 Milliseconds Vbucket 117 2016-04-07T03:44:15.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80310 Milliseconds Vbucket 117 2016-04-07T03:44:15.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80340 Milliseconds Vbucket 117 2016-04-07T03:44:15.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80370 Milliseconds Vbucket 117 2016-04-07T03:44:15.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80400 Milliseconds Vbucket 117 2016-04-07T03:44:15.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80430 Milliseconds Vbucket 117 2016-04-07T03:44:15.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80460 Milliseconds Vbucket 117 2016-04-07T03:44:15.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80490 Milliseconds Vbucket 117 2016-04-07T03:44:15.235-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7330000 QueuedCount 50000 2016-04-07T03:44:15.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80520 Milliseconds Vbucket 117 2016-04-07T03:44:15.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80550 Milliseconds Vbucket 117 2016-04-07T03:44:15.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80580 Milliseconds Vbucket 117 2016-04-07T03:44:15.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80610 Milliseconds Vbucket 117 2016-04-07T03:44:15.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80640 Milliseconds Vbucket 117 2016-04-07T03:44:15.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80670 Milliseconds Vbucket 117 2016-04-07T03:44:15.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80700 Milliseconds Vbucket 117 2016-04-07T03:44:15.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80730 Milliseconds Vbucket 117 2016-04-07T03:44:15.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80760 Milliseconds Vbucket 117 2016-04-07T03:44:15.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80790 Milliseconds Vbucket 117 2016-04-07T03:44:15.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80820 Milliseconds Vbucket 117 2016-04-07T03:44:15.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80850 Milliseconds Vbucket 117 2016-04-07T03:44:15.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80880 Milliseconds Vbucket 117 2016-04-07T03:44:15.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80910 Milliseconds Vbucket 117 2016-04-07T03:44:15.632-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7340000 QueuedCount 50000 2016-04-07T03:44:15.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80940 Milliseconds Vbucket 117 2016-04-07T03:44:15.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80970 Milliseconds Vbucket 117 2016-04-07T03:44:15.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81000 Milliseconds Vbucket 117 2016-04-07T03:44:15.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81030 Milliseconds Vbucket 117 2016-04-07T03:44:15.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81060 Milliseconds Vbucket 117 2016-04-07T03:44:15.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81090 Milliseconds Vbucket 117 2016-04-07T03:44:15.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81120 Milliseconds Vbucket 117 2016-04-07T03:44:15.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81150 Milliseconds Vbucket 117 2016-04-07T03:44:15.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81180 Milliseconds Vbucket 117 2016-04-07T03:44:15.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81210 Milliseconds Vbucket 117 2016-04-07T03:44:15.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81240 Milliseconds Vbucket 117 2016-04-07T03:44:15.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81270 Milliseconds Vbucket 117 2016-04-07T03:44:16.002-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7350000 QueuedCount 50000 2016-04-07T03:44:16.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81300 Milliseconds Vbucket 117 2016-04-07T03:44:16.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81330 Milliseconds Vbucket 117 2016-04-07T03:44:16.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81360 Milliseconds Vbucket 117 2016-04-07T03:44:16.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81390 Milliseconds Vbucket 117 2016-04-07T03:44:16.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81420 Milliseconds Vbucket 117 2016-04-07T03:44:16.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81450 Milliseconds Vbucket 117 2016-04-07T03:44:16.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81480 Milliseconds Vbucket 117 2016-04-07T03:44:16.222-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:16.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81510 Milliseconds Vbucket 117 2016-04-07T03:44:16.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81540 Milliseconds Vbucket 117 2016-04-07T03:44:16.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81570 Milliseconds Vbucket 117 2016-04-07T03:44:16.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81600 Milliseconds Vbucket 117 2016-04-07T03:44:16.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81630 Milliseconds Vbucket 117 2016-04-07T03:44:16.361-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7360000 QueuedCount 50000 2016-04-07T03:44:16.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81660 Milliseconds Vbucket 117 2016-04-07T03:44:16.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81690 Milliseconds Vbucket 117 2016-04-07T03:44:16.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81720 Milliseconds Vbucket 117 2016-04-07T03:44:16.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81750 Milliseconds Vbucket 117 2016-04-07T03:44:16.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81780 Milliseconds Vbucket 117 2016-04-07T03:44:16.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81810 Milliseconds Vbucket 117 2016-04-07T03:44:16.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81840 Milliseconds Vbucket 117 2016-04-07T03:44:16.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81870 Milliseconds Vbucket 117 2016-04-07T03:44:16.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81900 Milliseconds Vbucket 117 2016-04-07T03:44:16.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81930 Milliseconds Vbucket 117 2016-04-07T03:44:16.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81960 Milliseconds Vbucket 117 2016-04-07T03:44:16.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81990 Milliseconds Vbucket 117 2016-04-07T03:44:16.729-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7370000 QueuedCount 50000 2016-04-07T03:44:16.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82020 Milliseconds Vbucket 117 2016-04-07T03:44:16.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82050 Milliseconds Vbucket 117 2016-04-07T03:44:16.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82080 Milliseconds Vbucket 117 2016-04-07T03:44:16.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82110 Milliseconds Vbucket 117 2016-04-07T03:44:16.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82140 Milliseconds Vbucket 117 2016-04-07T03:44:16.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82170 Milliseconds Vbucket 117 2016-04-07T03:44:16.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82200 Milliseconds Vbucket 117 2016-04-07T03:44:16.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82230 Milliseconds Vbucket 117 2016-04-07T03:44:16.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82260 Milliseconds Vbucket 117 2016-04-07T03:44:17.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82290 Milliseconds Vbucket 117 2016-04-07T03:44:17.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82320 Milliseconds Vbucket 117 2016-04-07T03:44:17.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7380000 QueuedCount 50000 2016-04-07T03:44:17.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82350 Milliseconds Vbucket 117 2016-04-07T03:44:17.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82380 Milliseconds Vbucket 117 2016-04-07T03:44:17.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82410 Milliseconds Vbucket 117 2016-04-07T03:44:17.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82440 Milliseconds Vbucket 117 2016-04-07T03:44:17.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82470 Milliseconds Vbucket 117 2016-04-07T03:44:17.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82500 Milliseconds Vbucket 117 2016-04-07T03:44:17.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82530 Milliseconds Vbucket 117 2016-04-07T03:44:17.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82560 Milliseconds Vbucket 117 2016-04-07T03:44:17.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82590 Milliseconds Vbucket 117 2016-04-07T03:44:17.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82620 Milliseconds Vbucket 117 2016-04-07T03:44:17.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82650 Milliseconds Vbucket 117 2016-04-07T03:44:17.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82680 Milliseconds Vbucket 117 2016-04-07T03:44:17.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82710 Milliseconds Vbucket 117 2016-04-07T03:44:17.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82740 Milliseconds Vbucket 117 2016-04-07T03:44:17.463-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7390000 QueuedCount 50000 2016-04-07T03:44:17.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82770 Milliseconds Vbucket 117 2016-04-07T03:44:17.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82800 Milliseconds Vbucket 117 2016-04-07T03:44:17.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82830 Milliseconds Vbucket 117 2016-04-07T03:44:17.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82860 Milliseconds Vbucket 117 2016-04-07T03:44:17.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82890 Milliseconds Vbucket 117 2016-04-07T03:44:17.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82920 Milliseconds Vbucket 117 2016-04-07T03:44:17.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82950 Milliseconds Vbucket 117 2016-04-07T03:44:17.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82980 Milliseconds Vbucket 117 2016-04-07T03:44:17.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83010 Milliseconds Vbucket 117 2016-04-07T03:44:17.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83040 Milliseconds Vbucket 117 2016-04-07T03:44:17.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83070 Milliseconds Vbucket 117 2016-04-07T03:44:17.806-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7400000 QueuedCount 40971 2016-04-07T03:44:17.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83100 Milliseconds Vbucket 117 2016-04-07T03:44:17.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83130 Milliseconds Vbucket 117 2016-04-07T03:44:17.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83160 Milliseconds Vbucket 117 2016-04-07T03:44:17.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83190 Milliseconds Vbucket 117 2016-04-07T03:44:17.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83220 Milliseconds Vbucket 117 2016-04-07T03:44:17.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83250 Milliseconds Vbucket 117 2016-04-07T03:44:17.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83280 Milliseconds Vbucket 117 2016-04-07T03:44:18.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83310 Milliseconds Vbucket 117 2016-04-07T03:44:18.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83340 Milliseconds Vbucket 117 2016-04-07T03:44:18.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83370 Milliseconds Vbucket 117 2016-04-07T03:44:18.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83400 Milliseconds Vbucket 117 2016-04-07T03:44:18.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83430 Milliseconds Vbucket 117 2016-04-07T03:44:18.17-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7410000 QueuedCount 30971 2016-04-07T03:44:18.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83460 Milliseconds Vbucket 117 2016-04-07T03:44:18.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83490 Milliseconds Vbucket 117 2016-04-07T03:44:18.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83520 Milliseconds Vbucket 117 2016-04-07T03:44:18.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83550 Milliseconds Vbucket 117 2016-04-07T03:44:18.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83580 Milliseconds Vbucket 117 2016-04-07T03:44:18.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83610 Milliseconds Vbucket 117 2016-04-07T03:44:18.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83640 Milliseconds Vbucket 117 2016-04-07T03:44:18.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83670 Milliseconds Vbucket 117 2016-04-07T03:44:18.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83700 Milliseconds Vbucket 117 2016-04-07T03:44:18.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83730 Milliseconds Vbucket 117 2016-04-07T03:44:18.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83760 Milliseconds Vbucket 117 2016-04-07T03:44:18.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83790 Milliseconds Vbucket 117 2016-04-07T03:44:18.515-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7420000 QueuedCount 20971 2016-04-07T03:44:18.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83820 Milliseconds Vbucket 117 2016-04-07T03:44:18.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83850 Milliseconds Vbucket 117 2016-04-07T03:44:18.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83880 Milliseconds Vbucket 117 2016-04-07T03:44:18.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83910 Milliseconds Vbucket 117 2016-04-07T03:44:18.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83940 Milliseconds Vbucket 117 2016-04-07T03:44:18.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83970 Milliseconds Vbucket 117 2016-04-07T03:44:18.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84000 Milliseconds Vbucket 117 2016-04-07T03:44:18.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84030 Milliseconds Vbucket 117 2016-04-07T03:44:18.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84060 Milliseconds Vbucket 117 2016-04-07T03:44:18.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84090 Milliseconds Vbucket 117 2016-04-07T03:44:18.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84120 Milliseconds Vbucket 117 2016-04-07T03:44:18.84-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7430000 QueuedCount 10971 2016-04-07T03:44:18.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84150 Milliseconds Vbucket 117 2016-04-07T03:44:18.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84180 Milliseconds Vbucket 117 2016-04-07T03:44:18.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84210 Milliseconds Vbucket 117 2016-04-07T03:44:18.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84240 Milliseconds Vbucket 117 2016-04-07T03:44:18.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84270 Milliseconds Vbucket 117 2016-04-07T03:44:19.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84300 Milliseconds Vbucket 117 2016-04-07T03:44:19.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84330 Milliseconds Vbucket 117 2016-04-07T03:44:19.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84360 Milliseconds Vbucket 117 2016-04-07T03:44:19.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84390 Milliseconds Vbucket 117 2016-04-07T03:44:19.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84420 Milliseconds Vbucket 117 2016-04-07T03:44:19.159-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7440000 QueuedCount 971 2016-04-07T03:44:19.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84450 Milliseconds Vbucket 117 2016-04-07T03:44:19.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84480 Milliseconds Vbucket 117 2016-04-07T03:44:19.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84510 Milliseconds Vbucket 117 2016-04-07T03:44:19.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84540 Milliseconds Vbucket 117 2016-04-07T03:44:19.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84570 Milliseconds Vbucket 117 2016-04-07T03:44:19.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84600 Milliseconds Vbucket 117 2016-04-07T03:44:19.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84630 Milliseconds Vbucket 117 2016-04-07T03:44:19.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84660 Milliseconds Vbucket 117 2016-04-07T03:44:19.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84690 Milliseconds Vbucket 117 2016-04-07T03:44:19.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84720 Milliseconds Vbucket 117 2016-04-07T03:44:19.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84750 Milliseconds Vbucket 117 2016-04-07T03:44:19.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84780 Milliseconds Vbucket 117 2016-04-07T03:44:19.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84810 Milliseconds Vbucket 117 2016-04-07T03:44:19.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84840 Milliseconds Vbucket 117 2016-04-07T03:44:19.585-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.737µs CommitTime 401.634033ms TotalFlushTime 4m23.478659591s TotalCommitTime 12.813521258s 2016-04-07T03:44:19.586-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 7440971, 7440971, 49 committed:true 2016-04-07T03:44:19.586-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 7440971, 7440971, 49 committed:true) SnapCreateDur 411.972807ms SnapOpenDur 41.684µs 2016-04-07T03:44:19.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84870 Milliseconds Vbucket 117 2016-04-07T03:44:19.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84900 Milliseconds Vbucket 117 2016-04-07T03:44:19.632-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:19.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84930 Milliseconds Vbucket 117 2016-04-07T03:44:19.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84960 Milliseconds Vbucket 117 2016-04-07T03:44:19.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84990 Milliseconds Vbucket 117 2016-04-07T03:44:19.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85020 Milliseconds Vbucket 117 2016-04-07T03:44:19.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85050 Milliseconds Vbucket 117 2016-04-07T03:44:19.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85080 Milliseconds Vbucket 117 2016-04-07T03:44:19.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85110 Milliseconds Vbucket 117 2016-04-07T03:44:19.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85140 Milliseconds Vbucket 117 2016-04-07T03:44:19.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85170 Milliseconds Vbucket 117 2016-04-07T03:44:19.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85200 Milliseconds Vbucket 117 2016-04-07T03:44:19.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85230 Milliseconds Vbucket 117 2016-04-07T03:44:19.973-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7450000 QueuedCount 50000 2016-04-07T03:44:19.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85260 Milliseconds Vbucket 117 2016-04-07T03:44:20.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85290 Milliseconds Vbucket 117 2016-04-07T03:44:20.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85320 Milliseconds Vbucket 117 2016-04-07T03:44:20.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85350 Milliseconds Vbucket 117 2016-04-07T03:44:20.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85380 Milliseconds Vbucket 117 2016-04-07T03:44:20.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85410 Milliseconds Vbucket 117 2016-04-07T03:44:20.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85440 Milliseconds Vbucket 117 2016-04-07T03:44:20.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85470 Milliseconds Vbucket 117 2016-04-07T03:44:20.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85500 Milliseconds Vbucket 117 2016-04-07T03:44:20.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85530 Milliseconds Vbucket 117 2016-04-07T03:44:20.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85560 Milliseconds Vbucket 117 2016-04-07T03:44:20.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85590 Milliseconds Vbucket 117 2016-04-07T03:44:20.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85620 Milliseconds Vbucket 117 2016-04-07T03:44:20.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85650 Milliseconds Vbucket 117 2016-04-07T03:44:20.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85680 Milliseconds Vbucket 117 2016-04-07T03:44:20.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85710 Milliseconds Vbucket 117 2016-04-07T03:44:20.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85740 Milliseconds Vbucket 117 2016-04-07T03:44:20.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85770 Milliseconds Vbucket 117 2016-04-07T03:44:20.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85800 Milliseconds Vbucket 117 2016-04-07T03:44:20.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85830 Milliseconds Vbucket 117 2016-04-07T03:44:20.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85860 Milliseconds Vbucket 117 2016-04-07T03:44:20.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85890 Milliseconds Vbucket 117 2016-04-07T03:44:20.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85920 Milliseconds Vbucket 117 2016-04-07T03:44:20.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85950 Milliseconds Vbucket 117 2016-04-07T03:44:20.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85980 Milliseconds Vbucket 117 2016-04-07T03:44:20.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86010 Milliseconds Vbucket 117 2016-04-07T03:44:20.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86040 Milliseconds Vbucket 117 2016-04-07T03:44:20.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86070 Milliseconds Vbucket 117 2016-04-07T03:44:20.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86100 Milliseconds Vbucket 117 2016-04-07T03:44:20.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86130 Milliseconds Vbucket 117 2016-04-07T03:44:20.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86160 Milliseconds Vbucket 117 2016-04-07T03:44:20.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86190 Milliseconds Vbucket 117 2016-04-07T03:44:20.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86220 Milliseconds Vbucket 117 2016-04-07T03:44:20.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86250 Milliseconds Vbucket 117 2016-04-07T03:44:20.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86280 Milliseconds Vbucket 117 2016-04-07T03:44:21.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86310 Milliseconds Vbucket 117 2016-04-07T03:44:21.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86340 Milliseconds Vbucket 117 2016-04-07T03:44:21.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86370 Milliseconds Vbucket 117 2016-04-07T03:44:21.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86400 Milliseconds Vbucket 117 2016-04-07T03:44:21.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86430 Milliseconds Vbucket 117 2016-04-07T03:44:21.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86460 Milliseconds Vbucket 117 2016-04-07T03:44:21.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86490 Milliseconds Vbucket 117 2016-04-07T03:44:21.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86520 Milliseconds Vbucket 117 2016-04-07T03:44:21.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86550 Milliseconds Vbucket 117 2016-04-07T03:44:21.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86580 Milliseconds Vbucket 117 2016-04-07T03:44:21.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86610 Milliseconds Vbucket 117 2016-04-07T03:44:21.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86640 Milliseconds Vbucket 117 2016-04-07T03:44:21.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86670 Milliseconds Vbucket 117 2016-04-07T03:44:21.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86700 Milliseconds Vbucket 117 2016-04-07T03:44:21.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86730 Milliseconds Vbucket 117 2016-04-07T03:44:21.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86760 Milliseconds Vbucket 117 2016-04-07T03:44:21.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86790 Milliseconds Vbucket 117 2016-04-07T03:44:21.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86820 Milliseconds Vbucket 117 2016-04-07T03:44:21.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86850 Milliseconds Vbucket 117 2016-04-07T03:44:21.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86880 Milliseconds Vbucket 117 2016-04-07T03:44:21.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86910 Milliseconds Vbucket 117 2016-04-07T03:44:21.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86940 Milliseconds Vbucket 117 2016-04-07T03:44:21.672-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:21.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86970 Milliseconds Vbucket 117 2016-04-07T03:44:21.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87000 Milliseconds Vbucket 117 2016-04-07T03:44:21.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87030 Milliseconds Vbucket 117 2016-04-07T03:44:21.747-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7460000 QueuedCount 50000 2016-04-07T03:44:21.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87060 Milliseconds Vbucket 117 2016-04-07T03:44:21.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87090 Milliseconds Vbucket 117 2016-04-07T03:44:21.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87120 Milliseconds Vbucket 117 2016-04-07T03:44:21.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87150 Milliseconds Vbucket 117 2016-04-07T03:44:21.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87180 Milliseconds Vbucket 117 2016-04-07T03:44:21.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87210 Milliseconds Vbucket 117 2016-04-07T03:44:21.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87240 Milliseconds Vbucket 117 2016-04-07T03:44:21.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87270 Milliseconds Vbucket 117 2016-04-07T03:44:22.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87300 Milliseconds Vbucket 117 2016-04-07T03:44:22.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87330 Milliseconds Vbucket 117 2016-04-07T03:44:22.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87360 Milliseconds Vbucket 117 2016-04-07T03:44:22.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87390 Milliseconds Vbucket 117 2016-04-07T03:44:22.136-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7470000 QueuedCount 50000 2016-04-07T03:44:22.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87420 Milliseconds Vbucket 117 2016-04-07T03:44:22.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87450 Milliseconds Vbucket 117 2016-04-07T03:44:22.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87480 Milliseconds Vbucket 117 2016-04-07T03:44:22.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87510 Milliseconds Vbucket 117 2016-04-07T03:44:22.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87540 Milliseconds Vbucket 117 2016-04-07T03:44:22.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87570 Milliseconds Vbucket 117 2016-04-07T03:44:22.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87600 Milliseconds Vbucket 117 2016-04-07T03:44:22.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87630 Milliseconds Vbucket 117 2016-04-07T03:44:22.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87660 Milliseconds Vbucket 117 2016-04-07T03:44:22.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87690 Milliseconds Vbucket 117 2016-04-07T03:44:22.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87720 Milliseconds Vbucket 117 2016-04-07T03:44:22.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87750 Milliseconds Vbucket 117 2016-04-07T03:44:22.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87780 Milliseconds Vbucket 117 2016-04-07T03:44:22.509-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7480000 QueuedCount 50000 2016-04-07T03:44:22.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87810 Milliseconds Vbucket 117 2016-04-07T03:44:22.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87840 Milliseconds Vbucket 117 2016-04-07T03:44:22.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87870 Milliseconds Vbucket 117 2016-04-07T03:44:22.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87900 Milliseconds Vbucket 117 2016-04-07T03:44:22.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87930 Milliseconds Vbucket 117 2016-04-07T03:44:22.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87960 Milliseconds Vbucket 117 2016-04-07T03:44:22.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87990 Milliseconds Vbucket 117 2016-04-07T03:44:22.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88020 Milliseconds Vbucket 117 2016-04-07T03:44:22.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88050 Milliseconds Vbucket 117 2016-04-07T03:44:22.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88080 Milliseconds Vbucket 117 2016-04-07T03:44:22.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88110 Milliseconds Vbucket 117 2016-04-07T03:44:22.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88140 Milliseconds Vbucket 117 2016-04-07T03:44:22.866-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7490000 QueuedCount 50000 2016-04-07T03:44:22.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88170 Milliseconds Vbucket 117 2016-04-07T03:44:22.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88200 Milliseconds Vbucket 117 2016-04-07T03:44:22.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88230 Milliseconds Vbucket 117 2016-04-07T03:44:22.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88260 Milliseconds Vbucket 117 2016-04-07T03:44:23.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88290 Milliseconds Vbucket 117 2016-04-07T03:44:23.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88320 Milliseconds Vbucket 117 2016-04-07T03:44:23.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88350 Milliseconds Vbucket 117 2016-04-07T03:44:23.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88380 Milliseconds Vbucket 117 2016-04-07T03:44:23.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88410 Milliseconds Vbucket 117 2016-04-07T03:44:23.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88440 Milliseconds Vbucket 117 2016-04-07T03:44:23.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88470 Milliseconds Vbucket 117 2016-04-07T03:44:23.195-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7500000 QueuedCount 40343 2016-04-07T03:44:23.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88500 Milliseconds Vbucket 117 2016-04-07T03:44:23.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88530 Milliseconds Vbucket 117 2016-04-07T03:44:23.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88560 Milliseconds Vbucket 117 2016-04-07T03:44:23.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88590 Milliseconds Vbucket 117 2016-04-07T03:44:23.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88620 Milliseconds Vbucket 117 2016-04-07T03:44:23.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88650 Milliseconds Vbucket 117 2016-04-07T03:44:23.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88680 Milliseconds Vbucket 117 2016-04-07T03:44:23.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88710 Milliseconds Vbucket 117 2016-04-07T03:44:23.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88740 Milliseconds Vbucket 117 2016-04-07T03:44:23.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88770 Milliseconds Vbucket 117 2016-04-07T03:44:23.517-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7510000 QueuedCount 30343 2016-04-07T03:44:23.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88800 Milliseconds Vbucket 117 2016-04-07T03:44:23.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88830 Milliseconds Vbucket 117 2016-04-07T03:44:23.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88860 Milliseconds Vbucket 117 2016-04-07T03:44:23.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88890 Milliseconds Vbucket 117 2016-04-07T03:44:23.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88920 Milliseconds Vbucket 117 2016-04-07T03:44:23.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88950 Milliseconds Vbucket 117 2016-04-07T03:44:23.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88980 Milliseconds Vbucket 117 2016-04-07T03:44:23.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89010 Milliseconds Vbucket 117 2016-04-07T03:44:23.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89040 Milliseconds Vbucket 117 2016-04-07T03:44:23.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89070 Milliseconds Vbucket 117 2016-04-07T03:44:23.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89100 Milliseconds Vbucket 117 2016-04-07T03:44:23.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89130 Milliseconds Vbucket 117 2016-04-07T03:44:23.857-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7520000 QueuedCount 20343 2016-04-07T03:44:23.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89160 Milliseconds Vbucket 117 2016-04-07T03:44:23.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89190 Milliseconds Vbucket 117 2016-04-07T03:44:23.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89220 Milliseconds Vbucket 117 2016-04-07T03:44:23.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89250 Milliseconds Vbucket 117 2016-04-07T03:44:23.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89280 Milliseconds Vbucket 117 2016-04-07T03:44:24.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89310 Milliseconds Vbucket 117 2016-04-07T03:44:24.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89340 Milliseconds Vbucket 117 2016-04-07T03:44:24.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89370 Milliseconds Vbucket 117 2016-04-07T03:44:24.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89400 Milliseconds Vbucket 117 2016-04-07T03:44:24.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89430 Milliseconds Vbucket 117 2016-04-07T03:44:24.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89460 Milliseconds Vbucket 117 2016-04-07T03:44:24.205-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7530000 QueuedCount 10343 2016-04-07T03:44:24.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89490 Milliseconds Vbucket 117 2016-04-07T03:44:24.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89520 Milliseconds Vbucket 117 2016-04-07T03:44:24.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89550 Milliseconds Vbucket 117 2016-04-07T03:44:24.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89580 Milliseconds Vbucket 117 2016-04-07T03:44:24.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89610 Milliseconds Vbucket 117 2016-04-07T03:44:24.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89640 Milliseconds Vbucket 117 2016-04-07T03:44:24.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89670 Milliseconds Vbucket 117 2016-04-07T03:44:24.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89700 Milliseconds Vbucket 117 2016-04-07T03:44:24.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89730 Milliseconds Vbucket 117 2016-04-07T03:44:24.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89760 Milliseconds Vbucket 117 2016-04-07T03:44:24.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89790 Milliseconds Vbucket 117 2016-04-07T03:44:24.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89820 Milliseconds Vbucket 117 2016-04-07T03:44:24.54-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7540000 QueuedCount 343 2016-04-07T03:44:24.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89850 Milliseconds Vbucket 117 2016-04-07T03:44:24.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89880 Milliseconds Vbucket 117 2016-04-07T03:44:24.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89910 Milliseconds Vbucket 117 2016-04-07T03:44:24.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89940 Milliseconds Vbucket 117 2016-04-07T03:44:24.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89970 Milliseconds Vbucket 117 2016-04-07T03:44:24.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90000 Milliseconds Vbucket 117 2016-04-07T03:44:24.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90030 Milliseconds Vbucket 117 2016-04-07T03:44:24.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90060 Milliseconds Vbucket 117 2016-04-07T03:44:24.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90090 Milliseconds Vbucket 117 2016-04-07T03:44:24.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90120 Milliseconds Vbucket 117 2016-04-07T03:44:24.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90150 Milliseconds Vbucket 117 2016-04-07T03:44:24.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90180 Milliseconds Vbucket 117 2016-04-07T03:44:24.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90210 Milliseconds Vbucket 117 2016-04-07T03:44:24.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90240 Milliseconds Vbucket 117 2016-04-07T03:44:24.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90270 Milliseconds Vbucket 117 2016-04-07T03:44:25.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90300 Milliseconds Vbucket 117 2016-04-07T03:44:25.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90330 Milliseconds Vbucket 117 2016-04-07T03:44:25.061-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 13.941µs CommitTime 502.193398ms TotalFlushTime 4m28.295604381s TotalCommitTime 13.315714656s 2016-04-07T03:44:25.061-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 7540343, 7540343, 50 committed:true 2016-04-07T03:44:25.061-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 7540343, 7540343, 50 committed:true) SnapCreateDur 513.219213ms SnapOpenDur 34.895µs 2016-04-07T03:44:25.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90360 Milliseconds Vbucket 117 2016-04-07T03:44:25.102-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:25.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90390 Milliseconds Vbucket 117 2016-04-07T03:44:25.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90420 Milliseconds Vbucket 117 2016-04-07T03:44:25.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90450 Milliseconds Vbucket 117 2016-04-07T03:44:25.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90480 Milliseconds Vbucket 117 2016-04-07T03:44:25.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90510 Milliseconds Vbucket 117 2016-04-07T03:44:25.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90540 Milliseconds Vbucket 117 2016-04-07T03:44:25.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90570 Milliseconds Vbucket 117 2016-04-07T03:44:25.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90600 Milliseconds Vbucket 117 2016-04-07T03:44:25.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90630 Milliseconds Vbucket 117 2016-04-07T03:44:25.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90660 Milliseconds Vbucket 117 2016-04-07T03:44:25.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90690 Milliseconds Vbucket 117 2016-04-07T03:44:25.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90720 Milliseconds Vbucket 117 2016-04-07T03:44:25.457-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7550000 QueuedCount 50000 2016-04-07T03:44:25.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90750 Milliseconds Vbucket 117 2016-04-07T03:44:25.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90780 Milliseconds Vbucket 117 2016-04-07T03:44:25.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90810 Milliseconds Vbucket 117 2016-04-07T03:44:25.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90840 Milliseconds Vbucket 117 2016-04-07T03:44:25.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90870 Milliseconds Vbucket 117 2016-04-07T03:44:25.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90900 Milliseconds Vbucket 117 2016-04-07T03:44:25.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90930 Milliseconds Vbucket 117 2016-04-07T03:44:25.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90960 Milliseconds Vbucket 117 2016-04-07T03:44:25.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90990 Milliseconds Vbucket 117 2016-04-07T03:44:25.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91020 Milliseconds Vbucket 117 2016-04-07T03:44:25.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91050 Milliseconds Vbucket 117 2016-04-07T03:44:25.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91080 Milliseconds Vbucket 117 2016-04-07T03:44:25.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91110 Milliseconds Vbucket 117 2016-04-07T03:44:25.836-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:25.845-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7560000 QueuedCount 49199 2016-04-07T03:44:25.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91140 Milliseconds Vbucket 117 2016-04-07T03:44:25.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91170 Milliseconds Vbucket 117 2016-04-07T03:44:25.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91200 Milliseconds Vbucket 117 2016-04-07T03:44:25.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91230 Milliseconds Vbucket 117 2016-04-07T03:44:25.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91260 Milliseconds Vbucket 117 2016-04-07T03:44:26.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91290 Milliseconds Vbucket 117 2016-04-07T03:44:26.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91320 Milliseconds Vbucket 117 2016-04-07T03:44:26.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91350 Milliseconds Vbucket 117 2016-04-07T03:44:26.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91380 Milliseconds Vbucket 117 2016-04-07T03:44:26.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91410 Milliseconds Vbucket 117 2016-04-07T03:44:26.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91440 Milliseconds Vbucket 117 2016-04-07T03:44:26.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91470 Milliseconds Vbucket 117 2016-04-07T03:44:26.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91500 Milliseconds Vbucket 117 2016-04-07T03:44:26.229-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7570000 QueuedCount 50000 2016-04-07T03:44:26.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91530 Milliseconds Vbucket 117 2016-04-07T03:44:26.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91560 Milliseconds Vbucket 117 2016-04-07T03:44:26.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91590 Milliseconds Vbucket 117 2016-04-07T03:44:26.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91620 Milliseconds Vbucket 117 2016-04-07T03:44:26.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91650 Milliseconds Vbucket 117 2016-04-07T03:44:26.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91680 Milliseconds Vbucket 117 2016-04-07T03:44:26.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91710 Milliseconds Vbucket 117 2016-04-07T03:44:26.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91740 Milliseconds Vbucket 117 2016-04-07T03:44:26.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91770 Milliseconds Vbucket 117 2016-04-07T03:44:26.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91800 Milliseconds Vbucket 117 2016-04-07T03:44:26.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91830 Milliseconds Vbucket 117 2016-04-07T03:44:26.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91860 Milliseconds Vbucket 117 2016-04-07T03:44:26.601-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7580000 QueuedCount 50000 2016-04-07T03:44:26.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91890 Milliseconds Vbucket 117 2016-04-07T03:44:26.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91920 Milliseconds Vbucket 117 2016-04-07T03:44:26.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91950 Milliseconds Vbucket 117 2016-04-07T03:44:26.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91980 Milliseconds Vbucket 117 2016-04-07T03:44:26.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92010 Milliseconds Vbucket 117 2016-04-07T03:44:26.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92040 Milliseconds Vbucket 117 2016-04-07T03:44:26.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92070 Milliseconds Vbucket 117 2016-04-07T03:44:26.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92100 Milliseconds Vbucket 117 2016-04-07T03:44:26.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92130 Milliseconds Vbucket 117 2016-04-07T03:44:26.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92160 Milliseconds Vbucket 117 2016-04-07T03:44:26.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92190 Milliseconds Vbucket 117 2016-04-07T03:44:26.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92220 Milliseconds Vbucket 117 2016-04-07T03:44:26.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92250 Milliseconds Vbucket 117 2016-04-07T03:44:26.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92280 Milliseconds Vbucket 117 2016-04-07T03:44:26.999-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7590000 QueuedCount 50000 2016-04-07T03:44:27.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92310 Milliseconds Vbucket 117 2016-04-07T03:44:27.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92340 Milliseconds Vbucket 117 2016-04-07T03:44:27.08-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:27.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92370 Milliseconds Vbucket 117 2016-04-07T03:44:27.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92400 Milliseconds Vbucket 117 2016-04-07T03:44:27.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92430 Milliseconds Vbucket 117 2016-04-07T03:44:27.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92460 Milliseconds Vbucket 117 2016-04-07T03:44:27.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92490 Milliseconds Vbucket 117 2016-04-07T03:44:27.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92520 Milliseconds Vbucket 117 2016-04-07T03:44:27.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92550 Milliseconds Vbucket 117 2016-04-07T03:44:27.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92580 Milliseconds Vbucket 117 2016-04-07T03:44:27.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92610 Milliseconds Vbucket 117 2016-04-07T03:44:27.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92640 Milliseconds Vbucket 117 2016-04-07T03:44:27.384-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7600000 QueuedCount 50000 2016-04-07T03:44:27.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92670 Milliseconds Vbucket 117 2016-04-07T03:44:27.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92700 Milliseconds Vbucket 117 2016-04-07T03:44:27.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92730 Milliseconds Vbucket 117 2016-04-07T03:44:27.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92760 Milliseconds Vbucket 117 2016-04-07T03:44:27.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92790 Milliseconds Vbucket 117 2016-04-07T03:44:27.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92820 Milliseconds Vbucket 117 2016-04-07T03:44:27.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92850 Milliseconds Vbucket 117 2016-04-07T03:44:27.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92880 Milliseconds Vbucket 117 2016-04-07T03:44:27.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92910 Milliseconds Vbucket 117 2016-04-07T03:44:27.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92940 Milliseconds Vbucket 117 2016-04-07T03:44:27.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92970 Milliseconds Vbucket 117 2016-04-07T03:44:27.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93000 Milliseconds Vbucket 117 2016-04-07T03:44:27.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93030 Milliseconds Vbucket 117 2016-04-07T03:44:27.776-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7610000 QueuedCount 50000 2016-04-07T03:44:27.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93060 Milliseconds Vbucket 117 2016-04-07T03:44:27.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93090 Milliseconds Vbucket 117 2016-04-07T03:44:27.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93120 Milliseconds Vbucket 117 2016-04-07T03:44:27.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93150 Milliseconds Vbucket 117 2016-04-07T03:44:27.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93180 Milliseconds Vbucket 117 2016-04-07T03:44:27.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93210 Milliseconds Vbucket 117 2016-04-07T03:44:27.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93240 Milliseconds Vbucket 117 2016-04-07T03:44:27.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93270 Milliseconds Vbucket 117 2016-04-07T03:44:28.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93300 Milliseconds Vbucket 117 2016-04-07T03:44:28.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93330 Milliseconds Vbucket 117 2016-04-07T03:44:28.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93360 Milliseconds Vbucket 117 2016-04-07T03:44:28.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93390 Milliseconds Vbucket 117 2016-04-07T03:44:28.118-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7620000 QueuedCount 50000 2016-04-07T03:44:28.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93420 Milliseconds Vbucket 117 2016-04-07T03:44:28.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93450 Milliseconds Vbucket 117 2016-04-07T03:44:28.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93480 Milliseconds Vbucket 117 2016-04-07T03:44:28.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93510 Milliseconds Vbucket 117 2016-04-07T03:44:28.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93540 Milliseconds Vbucket 117 2016-04-07T03:44:28.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93570 Milliseconds Vbucket 117 2016-04-07T03:44:28.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93600 Milliseconds Vbucket 117 2016-04-07T03:44:28.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93630 Milliseconds Vbucket 117 2016-04-07T03:44:28.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93660 Milliseconds Vbucket 117 2016-04-07T03:44:28.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93690 Milliseconds Vbucket 117 2016-04-07T03:44:28.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93720 Milliseconds Vbucket 117 2016-04-07T03:44:28.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93750 Milliseconds Vbucket 117 2016-04-07T03:44:28.479-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7630000 QueuedCount 46063 2016-04-07T03:44:28.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93780 Milliseconds Vbucket 117 2016-04-07T03:44:28.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93810 Milliseconds Vbucket 117 2016-04-07T03:44:28.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93840 Milliseconds Vbucket 117 2016-04-07T03:44:28.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93870 Milliseconds Vbucket 117 2016-04-07T03:44:28.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93900 Milliseconds Vbucket 117 2016-04-07T03:44:28.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93930 Milliseconds Vbucket 117 2016-04-07T03:44:28.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93960 Milliseconds Vbucket 117 2016-04-07T03:44:28.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93990 Milliseconds Vbucket 117 2016-04-07T03:44:28.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94020 Milliseconds Vbucket 117 2016-04-07T03:44:28.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94050 Milliseconds Vbucket 117 2016-04-07T03:44:28.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94080 Milliseconds Vbucket 117 2016-04-07T03:44:28.816-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7640000 QueuedCount 36063 2016-04-07T03:44:28.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94110 Milliseconds Vbucket 117 2016-04-07T03:44:28.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94140 Milliseconds Vbucket 117 2016-04-07T03:44:28.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94170 Milliseconds Vbucket 117 2016-04-07T03:44:28.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94200 Milliseconds Vbucket 117 2016-04-07T03:44:28.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94230 Milliseconds Vbucket 117 2016-04-07T03:44:28.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94260 Milliseconds Vbucket 117 2016-04-07T03:44:29.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94290 Milliseconds Vbucket 117 2016-04-07T03:44:29.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94320 Milliseconds Vbucket 117 2016-04-07T03:44:29.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94350 Milliseconds Vbucket 117 2016-04-07T03:44:29.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94380 Milliseconds Vbucket 117 2016-04-07T03:44:29.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94410 Milliseconds Vbucket 117 2016-04-07T03:44:29.151-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7650000 QueuedCount 26063 2016-04-07T03:44:29.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94440 Milliseconds Vbucket 117 2016-04-07T03:44:29.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94470 Milliseconds Vbucket 117 2016-04-07T03:44:29.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94500 Milliseconds Vbucket 117 2016-04-07T03:44:29.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94530 Milliseconds Vbucket 117 2016-04-07T03:44:29.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94560 Milliseconds Vbucket 117 2016-04-07T03:44:29.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94590 Milliseconds Vbucket 117 2016-04-07T03:44:29.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94620 Milliseconds Vbucket 117 2016-04-07T03:44:29.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94650 Milliseconds Vbucket 117 2016-04-07T03:44:29.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94680 Milliseconds Vbucket 117 2016-04-07T03:44:29.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94710 Milliseconds Vbucket 117 2016-04-07T03:44:29.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94740 Milliseconds Vbucket 117 2016-04-07T03:44:29.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94770 Milliseconds Vbucket 117 2016-04-07T03:44:29.492-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7660000 QueuedCount 16063 2016-04-07T03:44:29.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94800 Milliseconds Vbucket 117 2016-04-07T03:44:29.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94830 Milliseconds Vbucket 117 2016-04-07T03:44:29.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94860 Milliseconds Vbucket 117 2016-04-07T03:44:29.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94890 Milliseconds Vbucket 117 2016-04-07T03:44:29.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94920 Milliseconds Vbucket 117 2016-04-07T03:44:29.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94950 Milliseconds Vbucket 117 2016-04-07T03:44:29.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94980 Milliseconds Vbucket 117 2016-04-07T03:44:29.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95010 Milliseconds Vbucket 117 2016-04-07T03:44:29.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95040 Milliseconds Vbucket 117 2016-04-07T03:44:29.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95070 Milliseconds Vbucket 117 2016-04-07T03:44:29.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95100 Milliseconds Vbucket 117 2016-04-07T03:44:29.829-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7670000 QueuedCount 6063 2016-04-07T03:44:29.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95130 Milliseconds Vbucket 117 2016-04-07T03:44:29.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95160 Milliseconds Vbucket 117 2016-04-07T03:44:29.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95190 Milliseconds Vbucket 117 2016-04-07T03:44:29.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95220 Milliseconds Vbucket 117 2016-04-07T03:44:29.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95250 Milliseconds Vbucket 117 2016-04-07T03:44:29.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95280 Milliseconds Vbucket 117 2016-04-07T03:44:30.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95310 Milliseconds Vbucket 117 2016-04-07T03:44:30.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95340 Milliseconds Vbucket 117 2016-04-07T03:44:30.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95370 Milliseconds Vbucket 117 2016-04-07T03:44:30.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95400 Milliseconds Vbucket 117 2016-04-07T03:44:30.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95430 Milliseconds Vbucket 117 2016-04-07T03:44:30.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95460 Milliseconds Vbucket 117 2016-04-07T03:44:30.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95490 Milliseconds Vbucket 117 2016-04-07T03:44:30.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95520 Milliseconds Vbucket 117 2016-04-07T03:44:30.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95550 Milliseconds Vbucket 117 2016-04-07T03:44:30.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95580 Milliseconds Vbucket 117 2016-04-07T03:44:30.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95610 Milliseconds Vbucket 117 2016-04-07T03:44:30.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95640 Milliseconds Vbucket 117 2016-04-07T03:44:30.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95670 Milliseconds Vbucket 117 2016-04-07T03:44:30.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95700 Milliseconds Vbucket 117 2016-04-07T03:44:30.435-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.098µs CommitTime 400.167979ms TotalFlushTime 4m33.033154057s TotalCommitTime 13.715882635s 2016-04-07T03:44:30.435-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 7676063, 7676063, 51 committed:true 2016-04-07T03:44:30.435-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 7676063, 7676063, 51 committed:true) SnapCreateDur 411.227236ms SnapOpenDur 49.668µs 2016-04-07T03:44:30.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95730 Milliseconds Vbucket 117 2016-04-07T03:44:30.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95760 Milliseconds Vbucket 117 2016-04-07T03:44:30.478-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:30.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95790 Milliseconds Vbucket 117 2016-04-07T03:44:30.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95820 Milliseconds Vbucket 117 2016-04-07T03:44:30.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95850 Milliseconds Vbucket 117 2016-04-07T03:44:30.583-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7680000 QueuedCount 50000 2016-04-07T03:44:30.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95880 Milliseconds Vbucket 117 2016-04-07T03:44:30.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95910 Milliseconds Vbucket 117 2016-04-07T03:44:30.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95940 Milliseconds Vbucket 117 2016-04-07T03:44:30.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95970 Milliseconds Vbucket 117 2016-04-07T03:44:30.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96000 Milliseconds Vbucket 117 2016-04-07T03:44:30.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96030 Milliseconds Vbucket 117 2016-04-07T03:44:30.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96060 Milliseconds Vbucket 117 2016-04-07T03:44:30.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96090 Milliseconds Vbucket 117 2016-04-07T03:44:30.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96120 Milliseconds Vbucket 117 2016-04-07T03:44:30.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96150 Milliseconds Vbucket 117 2016-04-07T03:44:30.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96180 Milliseconds Vbucket 117 2016-04-07T03:44:30.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96210 Milliseconds Vbucket 117 2016-04-07T03:44:30.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96240 Milliseconds Vbucket 117 2016-04-07T03:44:30.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96270 Milliseconds Vbucket 117 2016-04-07T03:44:30.999-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7690000 QueuedCount 50000 2016-04-07T03:44:31.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96300 Milliseconds Vbucket 117 2016-04-07T03:44:31.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96330 Milliseconds Vbucket 117 2016-04-07T03:44:31.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96360 Milliseconds Vbucket 117 2016-04-07T03:44:31.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96390 Milliseconds Vbucket 117 2016-04-07T03:44:31.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96420 Milliseconds Vbucket 117 2016-04-07T03:44:31.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96450 Milliseconds Vbucket 117 2016-04-07T03:44:31.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96480 Milliseconds Vbucket 117 2016-04-07T03:44:31.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96510 Milliseconds Vbucket 117 2016-04-07T03:44:31.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96540 Milliseconds Vbucket 117 2016-04-07T03:44:31.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96570 Milliseconds Vbucket 117 2016-04-07T03:44:31.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96600 Milliseconds Vbucket 117 2016-04-07T03:44:31.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96630 Milliseconds Vbucket 117 2016-04-07T03:44:31.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96660 Milliseconds Vbucket 117 2016-04-07T03:44:31.407-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7700000 QueuedCount 50000 2016-04-07T03:44:31.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96690 Milliseconds Vbucket 117 2016-04-07T03:44:31.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96720 Milliseconds Vbucket 117 2016-04-07T03:44:31.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96750 Milliseconds Vbucket 117 2016-04-07T03:44:31.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96780 Milliseconds Vbucket 117 2016-04-07T03:44:31.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96810 Milliseconds Vbucket 117 2016-04-07T03:44:31.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96840 Milliseconds Vbucket 117 2016-04-07T03:44:31.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96870 Milliseconds Vbucket 117 2016-04-07T03:44:31.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96900 Milliseconds Vbucket 117 2016-04-07T03:44:31.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96930 Milliseconds Vbucket 117 2016-04-07T03:44:31.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96960 Milliseconds Vbucket 117 2016-04-07T03:44:31.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96990 Milliseconds Vbucket 117 2016-04-07T03:44:31.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97020 Milliseconds Vbucket 117 2016-04-07T03:44:31.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97050 Milliseconds Vbucket 117 2016-04-07T03:44:31.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97080 Milliseconds Vbucket 117 2016-04-07T03:44:31.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97110 Milliseconds Vbucket 117 2016-04-07T03:44:31.836-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7710000 QueuedCount 50000 2016-04-07T03:44:31.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97140 Milliseconds Vbucket 117 2016-04-07T03:44:31.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97170 Milliseconds Vbucket 117 2016-04-07T03:44:31.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97200 Milliseconds Vbucket 117 2016-04-07T03:44:31.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97230 Milliseconds Vbucket 117 2016-04-07T03:44:31.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97260 Milliseconds Vbucket 117 2016-04-07T03:44:31.986-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:32.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97290 Milliseconds Vbucket 117 2016-04-07T03:44:32.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97320 Milliseconds Vbucket 117 2016-04-07T03:44:32.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97350 Milliseconds Vbucket 117 2016-04-07T03:44:32.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97380 Milliseconds Vbucket 117 2016-04-07T03:44:32.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97410 Milliseconds Vbucket 117 2016-04-07T03:44:32.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97440 Milliseconds Vbucket 117 2016-04-07T03:44:32.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97470 Milliseconds Vbucket 117 2016-04-07T03:44:32.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97500 Milliseconds Vbucket 117 2016-04-07T03:44:32.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97530 Milliseconds Vbucket 117 2016-04-07T03:44:32.249-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7720000 QueuedCount 50000 2016-04-07T03:44:32.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97560 Milliseconds Vbucket 117 2016-04-07T03:44:32.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97590 Milliseconds Vbucket 117 2016-04-07T03:44:32.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97620 Milliseconds Vbucket 117 2016-04-07T03:44:32.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97650 Milliseconds Vbucket 117 2016-04-07T03:44:32.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97680 Milliseconds Vbucket 117 2016-04-07T03:44:32.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97710 Milliseconds Vbucket 117 2016-04-07T03:44:32.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97740 Milliseconds Vbucket 117 2016-04-07T03:44:32.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97770 Milliseconds Vbucket 117 2016-04-07T03:44:32.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97800 Milliseconds Vbucket 117 2016-04-07T03:44:32.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97830 Milliseconds Vbucket 117 2016-04-07T03:44:32.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97860 Milliseconds Vbucket 117 2016-04-07T03:44:32.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97890 Milliseconds Vbucket 117 2016-04-07T03:44:32.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97920 Milliseconds Vbucket 117 2016-04-07T03:44:32.647-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7730000 QueuedCount 50000 2016-04-07T03:44:32.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97950 Milliseconds Vbucket 117 2016-04-07T03:44:32.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97980 Milliseconds Vbucket 117 2016-04-07T03:44:32.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98010 Milliseconds Vbucket 117 2016-04-07T03:44:32.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98040 Milliseconds Vbucket 117 2016-04-07T03:44:32.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98070 Milliseconds Vbucket 117 2016-04-07T03:44:32.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98100 Milliseconds Vbucket 117 2016-04-07T03:44:32.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98130 Milliseconds Vbucket 117 2016-04-07T03:44:32.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98160 Milliseconds Vbucket 117 2016-04-07T03:44:32.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98190 Milliseconds Vbucket 117 2016-04-07T03:44:32.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98220 Milliseconds Vbucket 117 2016-04-07T03:44:32.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98250 Milliseconds Vbucket 117 2016-04-07T03:44:32.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98280 Milliseconds Vbucket 117 2016-04-07T03:44:33.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98310 Milliseconds Vbucket 117 2016-04-07T03:44:33.03-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7740000 QueuedCount 50000 2016-04-07T03:44:33.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98340 Milliseconds Vbucket 117 2016-04-07T03:44:33.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98370 Milliseconds Vbucket 117 2016-04-07T03:44:33.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98400 Milliseconds Vbucket 117 2016-04-07T03:44:33.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98430 Milliseconds Vbucket 117 2016-04-07T03:44:33.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98460 Milliseconds Vbucket 117 2016-04-07T03:44:33.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98490 Milliseconds Vbucket 117 2016-04-07T03:44:33.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98520 Milliseconds Vbucket 117 2016-04-07T03:44:33.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98550 Milliseconds Vbucket 117 2016-04-07T03:44:33.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98580 Milliseconds Vbucket 117 2016-04-07T03:44:33.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98610 Milliseconds Vbucket 117 2016-04-07T03:44:33.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98640 Milliseconds Vbucket 117 2016-04-07T03:44:33.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98670 Milliseconds Vbucket 117 2016-04-07T03:44:33.396-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7750000 QueuedCount 50000 2016-04-07T03:44:33.404-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:33.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98700 Milliseconds Vbucket 117 2016-04-07T03:44:33.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98730 Milliseconds Vbucket 117 2016-04-07T03:44:33.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98760 Milliseconds Vbucket 117 2016-04-07T03:44:33.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98790 Milliseconds Vbucket 117 2016-04-07T03:44:33.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98820 Milliseconds Vbucket 117 2016-04-07T03:44:33.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98850 Milliseconds Vbucket 117 2016-04-07T03:44:33.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98880 Milliseconds Vbucket 117 2016-04-07T03:44:33.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98910 Milliseconds Vbucket 117 2016-04-07T03:44:33.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98940 Milliseconds Vbucket 117 2016-04-07T03:44:33.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98970 Milliseconds Vbucket 117 2016-04-07T03:44:33.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99000 Milliseconds Vbucket 117 2016-04-07T03:44:33.744-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7760000 QueuedCount 50000 2016-04-07T03:44:33.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99030 Milliseconds Vbucket 117 2016-04-07T03:44:33.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99060 Milliseconds Vbucket 117 2016-04-07T03:44:33.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99090 Milliseconds Vbucket 117 2016-04-07T03:44:33.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99120 Milliseconds Vbucket 117 2016-04-07T03:44:33.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99150 Milliseconds Vbucket 117 2016-04-07T03:44:34.242-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99180 Milliseconds Vbucket 117 2016-04-07T03:44:34.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99210 Milliseconds Vbucket 117 2016-04-07T03:44:34.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99240 Milliseconds Vbucket 117 2016-04-07T03:44:34.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99270 Milliseconds Vbucket 117 2016-04-07T03:44:34.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99300 Milliseconds Vbucket 117 2016-04-07T03:44:34.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99330 Milliseconds Vbucket 117 2016-04-07T03:44:34.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99360 Milliseconds Vbucket 117 2016-04-07T03:44:34.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99390 Milliseconds Vbucket 117 2016-04-07T03:44:34.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99420 Milliseconds Vbucket 117 2016-04-07T03:44:34.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99450 Milliseconds Vbucket 117 2016-04-07T03:44:34.514-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7770000 QueuedCount 50000 2016-04-07T03:44:34.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99480 Milliseconds Vbucket 117 2016-04-07T03:44:34.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99510 Milliseconds Vbucket 117 2016-04-07T03:44:34.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99540 Milliseconds Vbucket 117 2016-04-07T03:44:34.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99570 Milliseconds Vbucket 117 2016-04-07T03:44:34.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99600 Milliseconds Vbucket 117 2016-04-07T03:44:34.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99630 Milliseconds Vbucket 117 2016-04-07T03:44:34.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99660 Milliseconds Vbucket 117 2016-04-07T03:44:34.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99690 Milliseconds Vbucket 117 2016-04-07T03:44:34.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99720 Milliseconds Vbucket 117 2016-04-07T03:44:34.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99750 Milliseconds Vbucket 117 2016-04-07T03:44:34.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99780 Milliseconds Vbucket 117 2016-04-07T03:44:34.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99810 Milliseconds Vbucket 117 2016-04-07T03:44:34.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99840 Milliseconds Vbucket 117 2016-04-07T03:44:34.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99870 Milliseconds Vbucket 117 2016-04-07T03:44:34.93-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7780000 QueuedCount 50000 2016-04-07T03:44:34.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99900 Milliseconds Vbucket 117 2016-04-07T03:44:34.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99930 Milliseconds Vbucket 117 2016-04-07T03:44:35.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99960 Milliseconds Vbucket 117 2016-04-07T03:44:35.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99990 Milliseconds Vbucket 117 2016-04-07T03:44:35.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100020 Milliseconds Vbucket 117 2016-04-07T03:44:35.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100050 Milliseconds Vbucket 117 2016-04-07T03:44:35.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100080 Milliseconds Vbucket 117 2016-04-07T03:44:35.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100110 Milliseconds Vbucket 117 2016-04-07T03:44:35.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100140 Milliseconds Vbucket 117 2016-04-07T03:44:35.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100170 Milliseconds Vbucket 117 2016-04-07T03:44:35.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100200 Milliseconds Vbucket 117 2016-04-07T03:44:35.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100230 Milliseconds Vbucket 117 2016-04-07T03:44:35.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100260 Milliseconds Vbucket 117 2016-04-07T03:44:35.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100290 Milliseconds Vbucket 117 2016-04-07T03:44:35.341-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7790000 QueuedCount 47746 2016-04-07T03:44:35.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100320 Milliseconds Vbucket 117 2016-04-07T03:44:35.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100350 Milliseconds Vbucket 117 2016-04-07T03:44:35.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100380 Milliseconds Vbucket 117 2016-04-07T03:44:35.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100410 Milliseconds Vbucket 117 2016-04-07T03:44:35.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100440 Milliseconds Vbucket 117 2016-04-07T03:44:35.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100470 Milliseconds Vbucket 117 2016-04-07T03:44:35.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100500 Milliseconds Vbucket 117 2016-04-07T03:44:35.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100530 Milliseconds Vbucket 117 2016-04-07T03:44:35.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100560 Milliseconds Vbucket 117 2016-04-07T03:44:35.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100590 Milliseconds Vbucket 117 2016-04-07T03:44:35.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100620 Milliseconds Vbucket 117 2016-04-07T03:44:35.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100650 Milliseconds Vbucket 117 2016-04-07T03:44:35.724-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7800000 QueuedCount 37746 2016-04-07T03:44:35.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100680 Milliseconds Vbucket 117 2016-04-07T03:44:35.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100710 Milliseconds Vbucket 117 2016-04-07T03:44:35.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100740 Milliseconds Vbucket 117 2016-04-07T03:44:35.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100770 Milliseconds Vbucket 117 2016-04-07T03:44:35.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100800 Milliseconds Vbucket 117 2016-04-07T03:44:35.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100830 Milliseconds Vbucket 117 2016-04-07T03:44:35.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100860 Milliseconds Vbucket 117 2016-04-07T03:44:35.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100890 Milliseconds Vbucket 117 2016-04-07T03:44:35.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100920 Milliseconds Vbucket 117 2016-04-07T03:44:35.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100950 Milliseconds Vbucket 117 2016-04-07T03:44:36.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100980 Milliseconds Vbucket 117 2016-04-07T03:44:36.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101010 Milliseconds Vbucket 117 2016-04-07T03:44:36.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101040 Milliseconds Vbucket 117 2016-04-07T03:44:36.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101070 Milliseconds Vbucket 117 2016-04-07T03:44:36.123-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7810000 QueuedCount 27746 2016-04-07T03:44:36.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101100 Milliseconds Vbucket 117 2016-04-07T03:44:36.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101130 Milliseconds Vbucket 117 2016-04-07T03:44:36.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101160 Milliseconds Vbucket 117 2016-04-07T03:44:36.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101190 Milliseconds Vbucket 117 2016-04-07T03:44:36.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101220 Milliseconds Vbucket 117 2016-04-07T03:44:36.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101250 Milliseconds Vbucket 117 2016-04-07T03:44:36.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101280 Milliseconds Vbucket 117 2016-04-07T03:44:36.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101310 Milliseconds Vbucket 117 2016-04-07T03:44:36.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101340 Milliseconds Vbucket 117 2016-04-07T03:44:36.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101370 Milliseconds Vbucket 117 2016-04-07T03:44:36.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101400 Milliseconds Vbucket 117 2016-04-07T03:44:36.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101430 Milliseconds Vbucket 117 2016-04-07T03:44:36.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101460 Milliseconds Vbucket 117 2016-04-07T03:44:36.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101490 Milliseconds Vbucket 117 2016-04-07T03:44:36.554-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7820000 QueuedCount 17746 2016-04-07T03:44:36.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101520 Milliseconds Vbucket 117 2016-04-07T03:44:36.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101550 Milliseconds Vbucket 117 2016-04-07T03:44:36.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101580 Milliseconds Vbucket 117 2016-04-07T03:44:36.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101610 Milliseconds Vbucket 117 2016-04-07T03:44:36.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101640 Milliseconds Vbucket 117 2016-04-07T03:44:36.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101670 Milliseconds Vbucket 117 2016-04-07T03:44:36.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101700 Milliseconds Vbucket 117 2016-04-07T03:44:36.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101730 Milliseconds Vbucket 117 2016-04-07T03:44:36.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101760 Milliseconds Vbucket 117 2016-04-07T03:44:36.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101790 Milliseconds Vbucket 117 2016-04-07T03:44:36.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101820 Milliseconds Vbucket 117 2016-04-07T03:44:36.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101850 Milliseconds Vbucket 117 2016-04-07T03:44:36.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101880 Milliseconds Vbucket 117 2016-04-07T03:44:36.932-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7830000 QueuedCount 7746 2016-04-07T03:44:36.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101910 Milliseconds Vbucket 117 2016-04-07T03:44:36.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101940 Milliseconds Vbucket 117 2016-04-07T03:44:37.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101970 Milliseconds Vbucket 117 2016-04-07T03:44:37.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102000 Milliseconds Vbucket 117 2016-04-07T03:44:37.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102030 Milliseconds Vbucket 117 2016-04-07T03:44:37.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102060 Milliseconds Vbucket 117 2016-04-07T03:44:37.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102090 Milliseconds Vbucket 117 2016-04-07T03:44:37.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102120 Milliseconds Vbucket 117 2016-04-07T03:44:37.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102150 Milliseconds Vbucket 117 2016-04-07T03:44:37.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102180 Milliseconds Vbucket 117 2016-04-07T03:44:37.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102210 Milliseconds Vbucket 117 2016-04-07T03:44:37.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102240 Milliseconds Vbucket 117 2016-04-07T03:44:37.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102270 Milliseconds Vbucket 117 2016-04-07T03:44:37.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102300 Milliseconds Vbucket 117 2016-04-07T03:44:37.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102330 Milliseconds Vbucket 117 2016-04-07T03:44:37.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102360 Milliseconds Vbucket 117 2016-04-07T03:44:37.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102390 Milliseconds Vbucket 117 2016-04-07T03:44:37.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102420 Milliseconds Vbucket 117 2016-04-07T03:44:37.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102450 Milliseconds Vbucket 117 2016-04-07T03:44:37.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102480 Milliseconds Vbucket 117 2016-04-07T03:44:37.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102510 Milliseconds Vbucket 117 2016-04-07T03:44:37.558-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.241µs CommitTime 400.797284ms TotalFlushTime 4m39.473444879s TotalCommitTime 14.116679919s 2016-04-07T03:44:37.558-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 7837746, 7837746, 52 committed:true 2016-04-07T03:44:37.559-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 7837746, 7837746, 52 committed:true) SnapCreateDur 410.592243ms SnapOpenDur 52.707µs 2016-04-07T03:44:37.588-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.687121% blocked 2016-04-07T03:44:37.589-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.694647% blocked 2016-04-07T03:44:37.59-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.692265% blocked 2016-04-07T03:44:37.59-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.689584% blocked 2016-04-07T03:44:37.598-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14290000 2016-04-07T03:44:37.6-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:37.644-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14300000 2016-04-07T03:44:37.689-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7840000 QueuedCount 50000 2016-04-07T03:44:37.689-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14310000 2016-04-07T03:44:37.726-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14320000 2016-04-07T03:44:37.77-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14330000 2016-04-07T03:44:37.812-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14340000 2016-04-07T03:44:37.868-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14350000 2016-04-07T03:44:37.91-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14360000 2016-04-07T03:44:37.967-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14370000 2016-04-07T03:44:38.037-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14380000 2016-04-07T03:44:38.066-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7850000 QueuedCount 50000 2016-04-07T03:44:38.103-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14390000 2016-04-07T03:44:38.168-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14400000 2016-04-07T03:44:38.236-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14410000 2016-04-07T03:44:38.304-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14420000 2016-04-07T03:44:38.375-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14430000 2016-04-07T03:44:38.438-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14440000 2016-04-07T03:44:38.477-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:38.493-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7860000 QueuedCount 48569 2016-04-07T03:44:38.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14450000 2016-04-07T03:44:38.578-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14460000 2016-04-07T03:44:38.651-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14470000 2016-04-07T03:44:38.72-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14480000 2016-04-07T03:44:38.777-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14490000 2016-04-07T03:44:38.831-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14500000 2016-04-07T03:44:38.889-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14510000 2016-04-07T03:44:38.907-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7870000 QueuedCount 50000 2016-04-07T03:44:38.942-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14520000 2016-04-07T03:44:38.995-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14530000 2016-04-07T03:44:39.054-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14540000 2016-04-07T03:44:39.108-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14550000 2016-04-07T03:44:39.162-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14560000 2016-04-07T03:44:39.22-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14570000 2016-04-07T03:44:39.275-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14580000 2016-04-07T03:44:39.307-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7880000 QueuedCount 50000 2016-04-07T03:44:39.329-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14590000 2016-04-07T03:44:39.386-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14600000 2016-04-07T03:44:39.455-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14610000 2016-04-07T03:44:39.515-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14620000 2016-04-07T03:44:39.571-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14630000 2016-04-07T03:44:39.631-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14640000 2016-04-07T03:44:39.691-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14650000 2016-04-07T03:44:39.709-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7890000 QueuedCount 50000 2016-04-07T03:44:39.753-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14660000 2016-04-07T03:44:39.82-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14670000 2016-04-07T03:44:39.889-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14680000 2016-04-07T03:44:39.968-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14690000 2016-04-07T03:44:40.049-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14700000 2016-04-07T03:44:40.106-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7900000 QueuedCount 44839 2016-04-07T03:44:40.124-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14710000 2016-04-07T03:44:40.197-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14720000 2016-04-07T03:44:40.256-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14730000 2016-04-07T03:44:40.323-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14740000 2016-04-07T03:44:40.385-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14750000 2016-04-07T03:44:40.448-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14760000 2016-04-07T03:44:40.496-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7910000 QueuedCount 34839 2016-04-07T03:44:40.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14770000 2016-04-07T03:44:40.569-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14780000 2016-04-07T03:44:40.63-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14790000 2016-04-07T03:44:40.689-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14800000 2016-04-07T03:44:40.757-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14810000 2016-04-07T03:44:40.842-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14820000 2016-04-07T03:44:40.877-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7920000 QueuedCount 24839 2016-04-07T03:44:40.924-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14830000 2016-04-07T03:44:41.005-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14840000 2016-04-07T03:44:41.085-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14850000 2016-04-07T03:44:41.168-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14860000 2016-04-07T03:44:41.252-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14870000 2016-04-07T03:44:41.252-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7930000 QueuedCount 14839 2016-04-07T03:44:41.336-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14880000 2016-04-07T03:44:41.427-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14890000 2016-04-07T03:44:41.515-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14900000 2016-04-07T03:44:41.598-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14910000 2016-04-07T03:44:41.606-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7940000 QueuedCount 4839 2016-04-07T03:44:41.682-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14920000 2016-04-07T03:44:41.766-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14930000 2016-04-07T03:44:41.844-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14940000 2016-04-07T03:44:41.927-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14950000 2016-04-07T03:44:42.008-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14960000 2016-04-07T03:44:42.087-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14970000 2016-04-07T03:44:42.165-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14980000 2016-04-07T03:44:42.193-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.043µs CommitTime 349.840516ms TotalFlushTime 4m43.61394405s TotalCommitTime 14.466520435s 2016-04-07T03:44:42.193-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 7944839, 7944839, 53 committed:true 2016-04-07T03:44:42.193-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 7944839, 7944839, 53 committed:true) SnapCreateDur 360.797697ms SnapOpenDur 41.096µs 2016-04-07T03:44:42.237-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 14990000 2016-04-07T03:44:42.237-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:42.288-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15000000 2016-04-07T03:44:42.331-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15010000 2016-04-07T03:44:42.379-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15020000 2016-04-07T03:44:42.424-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7950000 QueuedCount 50000 2016-04-07T03:44:42.43-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15030000 2016-04-07T03:44:42.494-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15040000 2016-04-07T03:44:42.556-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15050000 2016-04-07T03:44:42.627-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15060000 2016-04-07T03:44:42.698-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15070000 2016-04-07T03:44:42.773-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15080000 2016-04-07T03:44:42.845-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15090000 2016-04-07T03:44:42.858-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7960000 QueuedCount 50000 2016-04-07T03:44:42.908-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15100000 2016-04-07T03:44:42.973-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15110000 2016-04-07T03:44:43.039-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15120000 2016-04-07T03:44:43.108-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15130000 2016-04-07T03:44:43.113-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:43.176-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15140000 2016-04-07T03:44:43.242-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15150000 2016-04-07T03:44:43.268-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7970000 QueuedCount 50000 2016-04-07T03:44:43.309-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15160000 2016-04-07T03:44:43.377-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15170000 2016-04-07T03:44:43.446-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15180000 2016-04-07T03:44:43.519-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15190000 2016-04-07T03:44:43.586-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15200000 2016-04-07T03:44:43.655-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15210000 2016-04-07T03:44:43.664-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7980000 QueuedCount 50000 2016-04-07T03:44:43.723-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15220000 2016-04-07T03:44:43.786-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15230000 2016-04-07T03:44:43.851-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15240000 2016-04-07T03:44:43.922-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15250000 2016-04-07T03:44:43.983-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15260000 2016-04-07T03:44:44.046-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15270000 2016-04-07T03:44:44.074-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 7990000 QueuedCount 50000 2016-04-07T03:44:44.107-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15280000 2016-04-07T03:44:44.166-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15290000 2016-04-07T03:44:44.222-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15300000 2016-04-07T03:44:44.278-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15310000 2016-04-07T03:44:44.336-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15320000 2016-04-07T03:44:44.394-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15330000 2016-04-07T03:44:44.428-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8000000 QueuedCount 50000 2016-04-07T03:44:44.455-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15340000 2016-04-07T03:44:44.49-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:44.523-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15350000 2016-04-07T03:44:44.593-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15360000 2016-04-07T03:44:44.665-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15370000 2016-04-07T03:44:44.736-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15380000 2016-04-07T03:44:44.807-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15390000 2016-04-07T03:44:44.824-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8010000 QueuedCount 50000 2016-04-07T03:44:44.878-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15400000 2016-04-07T03:44:44.945-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15410000 2016-04-07T03:44:45.012-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15420000 2016-04-07T03:44:45.079-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15430000 2016-04-07T03:44:45.148-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15440000 2016-04-07T03:44:45.218-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15450000 2016-04-07T03:44:45.223-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8020000 QueuedCount 50000 2016-04-07T03:44:45.289-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15460000 2016-04-07T03:44:45.361-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15470000 2016-04-07T03:44:45.427-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15480000 2016-04-07T03:44:45.496-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15490000 2016-04-07T03:44:45.564-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15500000 2016-04-07T03:44:45.611-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8030000 QueuedCount 50000 2016-04-07T03:44:45.632-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15510000 2016-04-07T03:44:45.701-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15520000 2016-04-07T03:44:45.769-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15530000 2016-04-07T03:44:45.845-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15540000 2016-04-07T03:44:45.915-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15550000 2016-04-07T03:44:45.983-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15560000 2016-04-07T03:44:45.992-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8040000 QueuedCount 44729 2016-04-07T03:44:46.059-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15570000 2016-04-07T03:44:46.134-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15580000 2016-04-07T03:44:46.208-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15590000 2016-04-07T03:44:46.284-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15600000 2016-04-07T03:44:46.358-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15610000 2016-04-07T03:44:46.369-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8050000 QueuedCount 34729 2016-04-07T03:44:46.428-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15620000 2016-04-07T03:44:46.501-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15630000 2016-04-07T03:44:46.576-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15640000 2016-04-07T03:44:46.652-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15650000 2016-04-07T03:44:46.727-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15660000 2016-04-07T03:44:46.754-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8060000 QueuedCount 24729 2016-04-07T03:44:46.795-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15670000 2016-04-07T03:44:46.873-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15680000 2016-04-07T03:44:46.951-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15690000 2016-04-07T03:44:47.034-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15700000 2016-04-07T03:44:47.114-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15710000 2016-04-07T03:44:47.138-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8070000 QueuedCount 14729 2016-04-07T03:44:47.192-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15720000 2016-04-07T03:44:47.269-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15730000 2016-04-07T03:44:47.344-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15740000 2016-04-07T03:44:47.425-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15750000 2016-04-07T03:44:47.461-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8080000 QueuedCount 4729 2016-04-07T03:44:47.502-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15760000 2016-04-07T03:44:47.58-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15770000 2016-04-07T03:44:47.658-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15780000 2016-04-07T03:44:47.74-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15790000 2016-04-07T03:44:47.817-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15800000 2016-04-07T03:44:47.898-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15810000 2016-04-07T03:44:47.976-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15820000 2016-04-07T03:44:48.055-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15830000 2016-04-07T03:44:48.08-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.031µs CommitTime 389.274079ms TotalFlushTime 4m48.914039191s TotalCommitTime 14.855794514s 2016-04-07T03:44:48.08-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 8084729, 8084729, 54 committed:true 2016-04-07T03:44:48.08-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 8084729, 8084729, 54 committed:true) SnapCreateDur 404.820924ms SnapOpenDur 47.848µs 2016-04-07T03:44:48.121-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15840000 2016-04-07T03:44:48.124-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:48.165-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15850000 2016-04-07T03:44:48.212-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15860000 2016-04-07T03:44:48.254-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15870000 2016-04-07T03:44:48.302-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15880000 2016-04-07T03:44:48.308-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8090000 QueuedCount 50000 2016-04-07T03:44:48.358-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15890000 2016-04-07T03:44:48.423-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15900000 2016-04-07T03:44:48.492-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15910000 2016-04-07T03:44:48.563-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15920000 2016-04-07T03:44:48.635-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15930000 2016-04-07T03:44:48.705-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15940000 2016-04-07T03:44:48.721-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8100000 QueuedCount 50000 2016-04-07T03:44:48.769-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15950000 2016-04-07T03:44:48.837-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15960000 2016-04-07T03:44:48.894-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:48.907-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15970000 2016-04-07T03:44:48.976-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15980000 2016-04-07T03:44:49.047-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 15990000 2016-04-07T03:44:49.12-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16000000 2016-04-07T03:44:49.131-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8110000 QueuedCount 50000 2016-04-07T03:44:49.188-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16010000 2016-04-07T03:44:49.253-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16020000 2016-04-07T03:44:49.316-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16030000 2016-04-07T03:44:49.382-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16040000 2016-04-07T03:44:49.45-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16050000 2016-04-07T03:44:49.522-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16060000 2016-04-07T03:44:49.529-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8120000 QueuedCount 50000 2016-04-07T03:44:49.589-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16070000 2016-04-07T03:44:49.655-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16080000 2016-04-07T03:44:49.719-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16090000 2016-04-07T03:44:49.786-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16100000 2016-04-07T03:44:49.851-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16110000 2016-04-07T03:44:49.924-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16120000 2016-04-07T03:44:49.927-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8130000 QueuedCount 50000 2016-04-07T03:44:49.992-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16130000 2016-04-07T03:44:50.064-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16140000 2016-04-07T03:44:50.133-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16150000 2016-04-07T03:44:50.2-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16160000 2016-04-07T03:44:50.266-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:50.273-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16170000 2016-04-07T03:44:50.284-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8140000 QueuedCount 48494 2016-04-07T03:44:50.346-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16180000 2016-04-07T03:44:50.414-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16190000 2016-04-07T03:44:50.481-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16200000 2016-04-07T03:44:50.546-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16210000 2016-04-07T03:44:50.613-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16220000 2016-04-07T03:44:50.683-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16230000 2016-04-07T03:44:50.688-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8150000 QueuedCount 50000 2016-04-07T03:44:50.754-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16240000 2016-04-07T03:44:50.824-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16250000 2016-04-07T03:44:50.896-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16260000 2016-04-07T03:44:50.969-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16270000 2016-04-07T03:44:51.038-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16280000 2016-04-07T03:44:51.109-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8160000 QueuedCount 50000 2016-04-07T03:44:51.11-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16290000 2016-04-07T03:44:51.181-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16300000 2016-04-07T03:44:51.258-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16310000 2016-04-07T03:44:51.328-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16320000 2016-04-07T03:44:51.398-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16330000 2016-04-07T03:44:51.473-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16340000 2016-04-07T03:44:51.546-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16350000 2016-04-07T03:44:51.55-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8170000 QueuedCount 50000 2016-04-07T03:44:51.62-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16360000 2016-04-07T03:44:51.701-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16370000 2016-04-07T03:44:51.791-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16380000 2016-04-07T03:44:51.88-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16390000 2016-04-07T03:44:51.938-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8180000 QueuedCount 41647 2016-04-07T03:44:51.963-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16400000 2016-04-07T03:44:52.048-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16410000 2016-04-07T03:44:52.128-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16420000 2016-04-07T03:44:52.211-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16430000 2016-04-07T03:44:52.292-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16440000 2016-04-07T03:44:52.317-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8190000 QueuedCount 31647 2016-04-07T03:44:52.375-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16450000 2016-04-07T03:44:52.454-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16460000 2016-04-07T03:44:52.535-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16470000 2016-04-07T03:44:52.615-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16480000 2016-04-07T03:44:52.694-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16490000 2016-04-07T03:44:52.695-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8200000 QueuedCount 21647 2016-04-07T03:44:52.774-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16500000 2016-04-07T03:44:52.853-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16510000 2016-04-07T03:44:52.932-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16520000 2016-04-07T03:44:53.013-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16530000 2016-04-07T03:44:53.069-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8210000 QueuedCount 11647 2016-04-07T03:44:53.091-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16540000 2016-04-07T03:44:53.167-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16550000 2016-04-07T03:44:53.247-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16560000 2016-04-07T03:44:53.327-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16570000 2016-04-07T03:44:53.41-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16580000 2016-04-07T03:44:53.447-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8220000 QueuedCount 1647 2016-04-07T03:44:53.905-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.359µs CommitTime 419.947392ms TotalFlushTime 4m54.131534928s TotalCommitTime 15.275741906s 2016-04-07T03:44:53.905-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 8221647, 8221647, 55 committed:true 2016-04-07T03:44:53.905-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 8221647, 8221647, 55 committed:true) SnapCreateDur 430.212172ms SnapOpenDur 38.825µs 2016-04-07T03:44:53.946-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:53.955-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16590000 2016-04-07T03:44:54.004-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16600000 2016-04-07T03:44:54.045-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16610000 2016-04-07T03:44:54.091-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16620000 2016-04-07T03:44:54.134-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16630000 2016-04-07T03:44:54.296-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8230000 QueuedCount 50000 2016-04-07T03:44:54.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16640000 2016-04-07T03:44:54.591-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:54.689-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8240000 QueuedCount 50000 2016-04-07T03:44:54.949-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16650000 2016-04-07T03:44:55.1-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8250000 QueuedCount 50000 2016-04-07T03:44:55.564-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8260000 QueuedCount 50000 2016-04-07T03:44:57.603-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:44:57.615-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8270000 QueuedCount 48770 2016-04-07T03:44:57.934-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16660000 2016-04-07T03:44:58.007-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8280000 QueuedCount 50000 2016-04-07T03:44:58.426-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8290000 QueuedCount 50000 2016-04-07T03:44:58.456-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16670000 2016-04-07T03:44:58.84-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8300000 QueuedCount 50000 2016-04-07T03:44:59.187-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8310000 QueuedCount 41815 2016-04-07T03:44:59.543-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8320000 QueuedCount 31815 2016-04-07T03:44:59.849-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8330000 QueuedCount 21815 2016-04-07T03:45:00.21-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8340000 QueuedCount 11815 2016-04-07T03:45:00.573-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8350000 QueuedCount 1815 2016-04-07T03:45:01.262-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.749µs CommitTime 607.748553ms TotalFlushTime 5m0.647193554s TotalCommitTime 15.883490459s 2016-04-07T03:45:01.262-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 8351815, 8351815, 56 committed:true 2016-04-07T03:45:01.262-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 8351815, 8351815, 56 committed:true) SnapCreateDur 618.417974ms SnapOpenDur 40.697µs 2016-04-07T03:45:01.303-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:01.317-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16680000 2016-04-07T03:45:01.362-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16690000 2016-04-07T03:45:01.406-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16700000 2016-04-07T03:45:01.453-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16710000 2016-04-07T03:45:01.499-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16720000 2016-04-07T03:45:01.547-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16730000 2016-04-07T03:45:01.596-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16740000 2016-04-07T03:45:01.603-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8360000 QueuedCount 50000 2016-04-07T03:45:01.65-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16750000 2016-04-07T03:45:01.878-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16760000 2016-04-07T03:45:02.039-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:02.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8370000 QueuedCount 49238 2016-04-07T03:45:02.162-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16770000 2016-04-07T03:45:02.478-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8380000 QueuedCount 50000 2016-04-07T03:45:02.655-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16780000 2016-04-07T03:45:02.885-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8390000 QueuedCount 50000 2016-04-07T03:45:03.054-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16790000 2016-04-07T03:45:03.246-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16800000 2016-04-07T03:45:03.297-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8400000 QueuedCount 50000 2016-04-07T03:45:03.804-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:05.162-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16810000 2016-04-07T03:45:05.293-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8410000 QueuedCount 50000 2016-04-07T03:45:05.479-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16820000 2016-04-07T03:45:05.687-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8420000 QueuedCount 50000 2016-04-07T03:45:05.882-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16830000 2016-04-07T03:45:05.995-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16840000 2016-04-07T03:45:06.147-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8430000 QueuedCount 50000 2016-04-07T03:45:06.585-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8440000 QueuedCount 49415 2016-04-07T03:45:06.874-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8450000 QueuedCount 39415 2016-04-07T03:45:07.196-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8460000 QueuedCount 29415 2016-04-07T03:45:07.527-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8470000 QueuedCount 19415 2016-04-07T03:45:07.847-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8480000 QueuedCount 9415 2016-04-07T03:45:08.634-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.743µs CommitTime 386.717862ms TotalFlushTime 5m7.389969952s TotalCommitTime 16.270208321s 2016-04-07T03:45:08.634-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 8489415, 8489415, 57 committed:true 2016-04-07T03:45:08.634-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 8489415, 8489415, 57 committed:true) SnapCreateDur 396.765174ms SnapOpenDur 48.42µs 2016-04-07T03:45:08.677-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:08.682-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16850000 2016-04-07T03:45:08.724-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16860000 2016-04-07T03:45:08.77-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16870000 2016-04-07T03:45:08.819-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16880000 2016-04-07T03:45:08.873-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16890000 2016-04-07T03:45:08.904-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8490000 QueuedCount 50000 2016-04-07T03:45:09.614-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8500000 QueuedCount 50000 2016-04-07T03:45:10.682-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8510000 QueuedCount 50000 2016-04-07T03:45:11.308-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:12.01-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16900000 2016-04-07T03:45:12.018-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8520000 QueuedCount 50000 2016-04-07T03:45:12.428-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8530000 QueuedCount 50000 2016-04-07T03:45:12.445-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16910000 2016-04-07T03:45:12.817-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16920000 2016-04-07T03:45:12.832-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8540000 QueuedCount 50000 2016-04-07T03:45:12.941-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8330709,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":6961991807,"bucket-1:myindex:avg_ts_items_count":135054,"bucket-1:myindex:build_progress":42,"bucket-1:myindex:data_size":2709721534,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":11413057536,"bucket-1:myindex:flush_queue_size":50018,"bucket-1:myindex:frag_percent":76,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":891388575,"bucket-1:myindex:items_count":8351815,"bucket-1:myindex:num_commits":57,"bucket-1:myindex:num_compactions":2,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":8453852,"bucket-1:myindex:num_docs_pending":3155421,"bucket-1:myindex:num_docs_queued":8390727,"bucket-1:myindex:num_flush_queued":8592693,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":57,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":7371372728,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"57 16270208321 5421036078899614331","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"8542676 54241547898 610559742510900758","bucket-1:myindex:timings/storage_handle_open":"57 1355900 33183924874","bucket-1:myindex:timings/storage_info":"229 2091140 48323269402","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"58 55891620 725168372755980","bucket-1:myindex:timings/storage_meta_set":"57 349511367 107823527446237745","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"17085351 251798158864 8976125857829858296","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":16923385,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":355,"memory_quota":31457280000,"memory_used":7388098560,"needs_restart":false,"num_connections":0,"timings/stats_response":"1772 182664830 29497345068426"} 2016-04-07T03:45:13.225-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8550000 QueuedCount 46340 2016-04-07T03:45:13.571-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8560000 QueuedCount 36340 2016-04-07T03:45:13.911-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8570000 QueuedCount 26340 2016-04-07T03:45:14.276-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8580000 QueuedCount 16340 2016-04-07T03:45:14.66-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8590000 QueuedCount 6340 2016-04-07T03:45:15.265-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.113µs CommitTime 376.421422ms TotalFlushTime 5m13.461763196s TotalCommitTime 16.646629743s 2016-04-07T03:45:15.265-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 8596340, 8596340, 58 committed:true 2016-04-07T03:45:15.265-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 8596340, 8596340, 58 committed:true) SnapCreateDur 387.135905ms SnapOpenDur 53.835µs 2016-04-07T03:45:15.307-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:15.316-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16930000 2016-04-07T03:45:15.359-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16940000 2016-04-07T03:45:15.406-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16950000 2016-04-07T03:45:15.422-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8600000 QueuedCount 50000 2016-04-07T03:45:15.448-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16960000 2016-04-07T03:45:15.527-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16970000 2016-04-07T03:45:15.723-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16980000 2016-04-07T03:45:15.873-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8610000 QueuedCount 50000 2016-04-07T03:45:15.888-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 16990000 2016-04-07T03:45:16.115-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:16.299-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8620000 QueuedCount 50000 2016-04-07T03:45:16.504-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17000000 2016-04-07T03:45:16.729-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8630000 QueuedCount 50000 2016-04-07T03:45:17.055-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17010000 2016-04-07T03:45:17.138-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8640000 QueuedCount 50000 2016-04-07T03:45:17.559-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8650000 QueuedCount 50000 2016-04-07T03:45:17.574-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:17.816-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17020000 2016-04-07T03:45:17.969-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8660000 QueuedCount 50000 2016-04-07T03:45:18.246-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17030000 2016-04-07T03:45:18.322-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8670000 QueuedCount 50000 2016-04-07T03:45:18.646-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17040000 2016-04-07T03:45:18.734-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8680000 QueuedCount 50000 2016-04-07T03:45:19.138-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8690000 QueuedCount 46553 2016-04-07T03:45:19.532-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8700000 QueuedCount 36553 2016-04-07T03:45:21.462-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8710000 QueuedCount 26553 2016-04-07T03:45:21.789-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8720000 QueuedCount 16553 2016-04-07T03:45:22.109-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8730000 QueuedCount 6553 2016-04-07T03:45:22.757-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.478µs CommitTime 435.561221ms TotalFlushTime 5m20.268720843s TotalCommitTime 17.082190964s 2016-04-07T03:45:22.757-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 8736553, 8736553, 59 committed:true 2016-04-07T03:45:22.757-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 8736553, 8736553, 59 committed:true) SnapCreateDur 445.403934ms SnapOpenDur 32.619µs 2016-04-07T03:45:22.802-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17050000 2016-04-07T03:45:22.847-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17060000 2016-04-07T03:45:22.891-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17070000 2016-04-07T03:45:22.899-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8740000 QueuedCount 29184 2016-04-07T03:45:23.34-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8750000 QueuedCount 19184 2016-04-07T03:45:23.728-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8760000 QueuedCount 9184 2016-04-07T03:45:24.349-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.362µs CommitTime 289.167168ms TotalFlushTime 5m21.515584902s TotalCommitTime 17.371358132s 2016-04-07T03:45:24.349-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 8769184, 8769184, 60 committed:true 2016-04-07T03:45:24.349-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 8769184, 8769184, 60 committed:true) SnapCreateDur 299.495898ms SnapOpenDur 43.103µs 2016-04-07T03:45:24.385-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8770000 QueuedCount 26494 2016-04-07T03:45:24.39-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:24.403-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17080000 2016-04-07T03:45:24.447-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17090000 2016-04-07T03:45:24.486-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17100000 2016-04-07T03:45:24.523-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17110000 2016-04-07T03:45:24.683-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17120000 2016-04-07T03:45:24.835-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8780000 QueuedCount 50000 2016-04-07T03:45:25.011-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:25.256-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8790000 QueuedCount 50000 2016-04-07T03:45:25.379-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17130000 2016-04-07T03:45:25.674-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8800000 QueuedCount 50000 2016-04-07T03:45:25.808-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17140000 2016-04-07T03:45:26.032-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8810000 QueuedCount 50000 2016-04-07T03:45:26.208-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17150000 2016-04-07T03:45:26.35-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:26.441-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8820000 QueuedCount 50000 2016-04-07T03:45:26.611-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17160000 2016-04-07T03:45:26.865-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8830000 QueuedCount 50000 2016-04-07T03:45:27.128-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17170000 2016-04-07T03:45:27.273-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8840000 QueuedCount 50000 2016-04-07T03:45:27.38-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17180000 2016-04-07T03:45:27.543-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17190000 2016-04-07T03:45:27.667-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:27.668-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8850000 QueuedCount 49873 2016-04-07T03:45:27.916-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17200000 2016-04-07T03:45:28.057-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8860000 QueuedCount 50000 2016-04-07T03:45:28.32-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17210000 2016-04-07T03:45:28.448-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8870000 QueuedCount 50000 2016-04-07T03:45:28.628-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17220000 2016-04-07T03:45:28.841-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8880000 QueuedCount 50000 2016-04-07T03:45:28.868-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:28.939-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17230000 2016-04-07T03:45:29.191-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8890000 QueuedCount 50000 2016-04-07T03:45:29.489-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17240000 2016-04-07T03:45:29.571-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8900000 QueuedCount 50000 2016-04-07T03:45:30.005-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8910000 QueuedCount 50000 2016-04-07T03:45:30.398-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8920000 QueuedCount 44878 2016-04-07T03:45:30.763-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8930000 QueuedCount 34878 2016-04-07T03:45:31.135-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8940000 QueuedCount 24878 2016-04-07T03:45:31.497-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8950000 QueuedCount 14878 2016-04-07T03:45:31.86-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8960000 QueuedCount 4878 2016-04-07T03:45:32.553-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.708µs CommitTime 513.863256ms TotalFlushTime 5m28.849100992s TotalCommitTime 17.885221388s 2016-04-07T03:45:32.553-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 8964878, 8964878, 61 committed:true 2016-04-07T03:45:32.553-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 8964878, 8964878, 61 committed:true) SnapCreateDur 524.309514ms SnapOpenDur 43.915µs 2016-04-07T03:45:32.59-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17250000 2016-04-07T03:45:32.594-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:32.63-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17260000 2016-04-07T03:45:32.674-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17270000 2016-04-07T03:45:32.716-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17280000 2016-04-07T03:45:32.792-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17290000 2016-04-07T03:45:34.215-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8970000 QueuedCount 50000 2016-04-07T03:45:34.663-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8980000 QueuedCount 50000 2016-04-07T03:45:35.012-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:35.084-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 8990000 QueuedCount 50000 2016-04-07T03:45:35.298-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17300000 2016-04-07T03:45:35.511-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9000000 QueuedCount 50000 2016-04-07T03:45:35.882-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17310000 2016-04-07T03:45:35.938-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9010000 QueuedCount 50000 2016-04-07T03:45:36.309-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9020000 QueuedCount 50000 2016-04-07T03:45:36.315-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17320000 2016-04-07T03:45:36.566-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17330000 2016-04-07T03:45:36.725-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9030000 QueuedCount 50000 2016-04-07T03:45:36.82-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17340000 2016-04-07T03:45:36.955-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17350000 2016-04-07T03:45:37.181-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9040000 QueuedCount 44528 2016-04-07T03:45:37.564-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9050000 QueuedCount 34528 2016-04-07T03:45:37.924-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9060000 QueuedCount 24528 2016-04-07T03:45:38.28-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9070000 QueuedCount 14528 2016-04-07T03:45:38.634-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9080000 QueuedCount 4528 2016-04-07T03:45:39.248-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.547µs CommitTime 436.614184ms TotalFlushTime 5m34.889407681s TotalCommitTime 18.321835572s 2016-04-07T03:45:39.248-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 9084528, 9084528, 62 committed:true 2016-04-07T03:45:39.248-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 9084528, 9084528, 62 committed:true) SnapCreateDur 447.297443ms SnapOpenDur 34.884µs 2016-04-07T03:45:39.292-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:39.313-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17360000 2016-04-07T03:45:39.358-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17370000 2016-04-07T03:45:39.396-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17380000 2016-04-07T03:45:39.437-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17390000 2016-04-07T03:45:39.481-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17400000 2016-04-07T03:45:39.484-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9090000 QueuedCount 50000 2016-04-07T03:45:39.49-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:39.525-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17410000 2016-04-07T03:45:39.865-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17420000 2016-04-07T03:45:39.936-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9100000 QueuedCount 50000 2016-04-07T03:45:40.008-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:40.203-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17430000 2016-04-07T03:45:40.356-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9110000 QueuedCount 50000 2016-04-07T03:45:40.57-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17440000 2016-04-07T03:45:40.616-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17450000 2016-04-07T03:45:40.655-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17460000 2016-04-07T03:45:40.7-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17470000 2016-04-07T03:45:40.746-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17480000 2016-04-07T03:45:40.788-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17490000 2016-04-07T03:45:40.827-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9120000 QueuedCount 50000 2016-04-07T03:45:40.832-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17500000 2016-04-07T03:45:40.909-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17510000 2016-04-07T03:45:41.24-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9130000 QueuedCount 50000 2016-04-07T03:45:41.249-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17520000 2016-04-07T03:45:41.414-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:41.667-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9140000 QueuedCount 50000 2016-04-07T03:45:41.714-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17530000 2016-04-07T03:45:43.582-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9150000 QueuedCount 50000 2016-04-07T03:45:43.593-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17540000 2016-04-07T03:45:43.911-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17550000 2016-04-07T03:45:43.987-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9160000 QueuedCount 50000 2016-04-07T03:45:44.402-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9170000 QueuedCount 50000 2016-04-07T03:45:44.778-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9180000 QueuedCount 41137 2016-04-07T03:45:45.148-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9190000 QueuedCount 31137 2016-04-07T03:45:45.564-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9200000 QueuedCount 21137 2016-04-07T03:45:45.921-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9210000 QueuedCount 11137 2016-04-07T03:45:46.277-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9220000 QueuedCount 1137 2016-04-07T03:45:46.762-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 17.838µs CommitTime 461.053651ms TotalFlushTime 5m41.705171798s TotalCommitTime 18.782889223s 2016-04-07T03:45:46.762-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 9221137, 9221137, 63 committed:true 2016-04-07T03:45:46.762-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 9221137, 9221137, 63 committed:true) SnapCreateDur 472.011396ms SnapOpenDur 48.875µs 2016-04-07T03:45:46.803-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17560000 2016-04-07T03:45:46.851-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17570000 2016-04-07T03:45:46.904-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17580000 2016-04-07T03:45:46.96-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17590000 2016-04-07T03:45:47.158-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9230000 QueuedCount 26862 2016-04-07T03:45:47.561-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9240000 QueuedCount 16862 2016-04-07T03:45:47.917-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9250000 QueuedCount 6862 2016-04-07T03:45:48.429-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.912µs CommitTime 283.13229ms TotalFlushTime 5m43.033564132s TotalCommitTime 19.066021513s 2016-04-07T03:45:48.429-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 9256862, 9256862, 64 committed:true 2016-04-07T03:45:48.429-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 9256862, 9256862, 64 committed:true) SnapCreateDur 292.8758ms SnapOpenDur 45.012µs 2016-04-07T03:45:48.473-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:48.498-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17600000 2016-04-07T03:45:48.543-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17610000 2016-04-07T03:45:48.572-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9260000 QueuedCount 50000 2016-04-07T03:45:48.583-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17620000 2016-04-07T03:45:48.662-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17630000 2016-04-07T03:45:49.03-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9270000 QueuedCount 50000 2016-04-07T03:45:49.065-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17640000 2016-04-07T03:45:49.317-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17650000 2016-04-07T03:45:49.696-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17660000 2016-04-07T03:45:49.742-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:49.747-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9280000 QueuedCount 49566 2016-04-07T03:45:49.751-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17670000 2016-04-07T03:45:50.124-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17680000 2016-04-07T03:45:50.201-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9290000 QueuedCount 50000 2016-04-07T03:45:51.152-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17690000 2016-04-07T03:45:51.73-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9300000 QueuedCount 50000 2016-04-07T03:45:52.607-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17700000 2016-04-07T03:45:52.711-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9310000 QueuedCount 50000 2016-04-07T03:45:52.977-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:53.143-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9320000 QueuedCount 50000 2016-04-07T03:45:53.25-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17710000 2016-04-07T03:45:53.561-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9330000 QueuedCount 50000 2016-04-07T03:45:53.743-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17720000 2016-04-07T03:45:53.991-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9340000 QueuedCount 50000 2016-04-07T03:45:54.174-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17730000 2016-04-07T03:45:54.367-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17740000 2016-04-07T03:45:54.412-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9350000 QueuedCount 50000 2016-04-07T03:45:54.507-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17750000 2016-04-07T03:45:54.807-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9360000 QueuedCount 42147 2016-04-07T03:45:55.197-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9370000 QueuedCount 32147 2016-04-07T03:45:55.585-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9380000 QueuedCount 22147 2016-04-07T03:45:55.898-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9390000 QueuedCount 12147 2016-04-07T03:45:56.264-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9400000 QueuedCount 2147 2016-04-07T03:45:56.817-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.04µs CommitTime 462.225033ms TotalFlushTime 5m50.685351457s TotalCommitTime 19.528246546s 2016-04-07T03:45:56.817-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 9402147, 9402147, 65 committed:true 2016-04-07T03:45:56.817-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 9402147, 9402147, 65 committed:true) SnapCreateDur 472.612854ms SnapOpenDur 45.33µs 2016-04-07T03:45:56.862-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:56.866-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17760000 2016-04-07T03:45:56.905-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17770000 2016-04-07T03:45:56.947-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17780000 2016-04-07T03:45:56.986-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17790000 2016-04-07T03:45:57.04-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17800000 2016-04-07T03:45:57.155-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9410000 QueuedCount 50000 2016-04-07T03:45:57.235-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17810000 2016-04-07T03:45:57.611-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9420000 QueuedCount 50000 2016-04-07T03:45:57.781-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:57.966-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17820000 2016-04-07T03:45:58.028-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9430000 QueuedCount 50000 2016-04-07T03:45:58.278-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17830000 2016-04-07T03:45:58.413-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9440000 QueuedCount 50000 2016-04-07T03:45:58.532-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17840000 2016-04-07T03:45:58.841-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9450000 QueuedCount 50000 2016-04-07T03:45:59.257-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9460000 QueuedCount 50000 2016-04-07T03:45:59.263-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:45:59.535-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17850000 2016-04-07T03:45:59.664-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9470000 QueuedCount 50000 2016-04-07T03:46:00.091-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9480000 QueuedCount 50000 2016-04-07T03:46:00.488-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9490000 QueuedCount 50000 2016-04-07T03:46:00.893-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9500000 QueuedCount 47539 2016-04-07T03:46:01.247-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9510000 QueuedCount 37539 2016-04-07T03:46:01.599-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9520000 QueuedCount 27539 2016-04-07T03:46:01.964-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9530000 QueuedCount 17539 2016-04-07T03:46:02.304-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9540000 QueuedCount 7539 2016-04-07T03:46:02.997-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.888µs CommitTime 466.51372ms TotalFlushTime 5m56.125597511s TotalCommitTime 19.994760266s 2016-04-07T03:46:02.997-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 9547539, 9547539, 66 committed:true 2016-04-07T03:46:02.997-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 9547539, 9547539, 66 committed:true) SnapCreateDur 476.48807ms SnapOpenDur 55.329µs 2016-04-07T03:46:03.043-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:03.056-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17860000 2016-04-07T03:46:03.094-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17870000 2016-04-07T03:46:03.13-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9550000 QueuedCount 50000 2016-04-07T03:46:03.141-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17880000 2016-04-07T03:46:03.337-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17890000 2016-04-07T03:46:03.446-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17900000 2016-04-07T03:46:03.564-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9560000 QueuedCount 50000 2016-04-07T03:46:03.839-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:03.923-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9570000 QueuedCount 50000 2016-04-07T03:46:04.27-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17910000 2016-04-07T03:46:04.334-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9580000 QueuedCount 50000 2016-04-07T03:46:04.742-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9590000 QueuedCount 50000 2016-04-07T03:46:05.147-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9600000 QueuedCount 50000 2016-04-07T03:46:05.154-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:05.304-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17920000 2016-04-07T03:46:05.375-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17930000 2016-04-07T03:46:05.418-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17940000 2016-04-07T03:46:05.53-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9610000 QueuedCount 50000 2016-04-07T03:46:05.91-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9620000 QueuedCount 50000 2016-04-07T03:46:06.312-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9630000 QueuedCount 50000 2016-04-07T03:46:06.685-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9640000 QueuedCount 44953 2016-04-07T03:46:07.053-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9650000 QueuedCount 34953 2016-04-07T03:46:07.357-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9660000 QueuedCount 24953 2016-04-07T03:46:07.701-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9670000 QueuedCount 14953 2016-04-07T03:46:08.058-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9680000 QueuedCount 4953 2016-04-07T03:46:08.689-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.403µs CommitTime 406.15842ms TotalFlushTime 6m1.17348788s TotalCommitTime 20.400918686s 2016-04-07T03:46:08.689-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 9684953, 9684953, 67 committed:true 2016-04-07T03:46:08.689-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 9684953, 9684953, 67 committed:true) SnapCreateDur 416.560174ms SnapOpenDur 42.872µs 2016-04-07T03:46:08.73-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:10.455-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 562 2016-04-07T03:46:10.485-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 562 2016-04-07T03:46:10.515-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 562 2016-04-07T03:46:10.545-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 562 2016-04-07T03:46:10.562-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9690000 QueuedCount 50000 2016-04-07T03:46:10.575-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 562 2016-04-07T03:46:10.605-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 562 2016-04-07T03:46:10.635-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 562 2016-04-07T03:46:10.665-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 562 2016-04-07T03:46:10.695-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 562 2016-04-07T03:46:10.725-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 562 2016-04-07T03:46:10.755-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 562 2016-04-07T03:46:10.785-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 562 2016-04-07T03:46:10.815-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 562 2016-04-07T03:46:10.845-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 562 2016-04-07T03:46:10.875-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 562 2016-04-07T03:46:10.905-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 562 2016-04-07T03:46:10.935-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 562 2016-04-07T03:46:10.965-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 562 2016-04-07T03:46:10.995-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 562 2016-04-07T03:46:11.025-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 562 2016-04-07T03:46:11.042-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9700000 QueuedCount 50000 2016-04-07T03:46:11.055-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 562 2016-04-07T03:46:11.085-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 562 2016-04-07T03:46:11.115-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 562 2016-04-07T03:46:11.133-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:11.145-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 562 2016-04-07T03:46:11.464-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9710000 QueuedCount 50000 2016-04-07T03:46:11.886-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9720000 QueuedCount 50000 2016-04-07T03:46:12.288-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9730000 QueuedCount 50000 2016-04-07T03:46:12.698-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9740000 QueuedCount 46077 2016-04-07T03:46:12.941-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8160126,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":5981060526,"bucket-1:myindex:avg_ts_items_count":133753,"bucket-1:myindex:build_progress":48,"bucket-1:myindex:data_size":3100059628,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":14235238400,"bucket-1:myindex:flush_queue_size":39670,"bucket-1:myindex:frag_percent":78,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":1016920065,"bucket-1:myindex:items_count":9547539,"bucket-1:myindex:num_commits":67,"bucket-1:myindex:num_compactions":2,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":9650502,"bucket-1:myindex:num_docs_pending":2054144,"bucket-1:myindex:num_docs_queued":8295354,"bucket-1:myindex:num_flush_queued":9786077,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":67,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":5691576644,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"67 20400918686 7179733842623234662","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"9746408 61938947655 610612613518086819","bucket-1:myindex:timings/storage_handle_open":"67 1578411 38286945067","bucket-1:myindex:timings/storage_info":"269 2504036 58447367148","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"68 59169920 726268450286868","bucket-1:myindex:timings/storage_meta_set":"67 353404241 107825055552589335","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"19492815 297025417488 -7689820068719420484","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":17946203,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":469,"memory_quota":31457280000,"memory_used":11311788032,"needs_restart":false,"num_connections":0,"timings/stats_response":"1915 208314702 34230770900284"} 2016-04-07T03:46:13.053-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9750000 QueuedCount 36077 2016-04-07T03:46:13.449-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9760000 QueuedCount 26077 2016-04-07T03:46:13.78-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9770000 QueuedCount 16077 2016-04-07T03:46:14.114-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9780000 QueuedCount 6077 2016-04-07T03:46:14.746-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.217µs CommitTime 418.577738ms TotalFlushTime 6m6.639913441s TotalCommitTime 20.819496424s 2016-04-07T03:46:14.747-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 9786077, 9786077, 68 committed:true 2016-04-07T03:46:14.747-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 9786077, 9786077, 68 committed:true) SnapCreateDur 429.747862ms SnapOpenDur 74.455µs 2016-04-07T03:46:14.789-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:14.856-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17950000 2016-04-07T03:46:14.889-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9790000 QueuedCount 50000 2016-04-07T03:46:14.895-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17960000 2016-04-07T03:46:14.943-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17970000 2016-04-07T03:46:14.981-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17980000 2016-04-07T03:46:15.028-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 17990000 2016-04-07T03:46:15.07-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18000000 2016-04-07T03:46:15.332-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9800000 QueuedCount 50000 2016-04-07T03:46:15.503-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:15.625-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18010000 2016-04-07T03:46:15.664-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18020000 2016-04-07T03:46:15.735-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9810000 QueuedCount 50000 2016-04-07T03:46:15.768-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18030000 2016-04-07T03:46:15.808-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18040000 2016-04-07T03:46:15.845-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18050000 2016-04-07T03:46:16.117-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9820000 QueuedCount 50000 2016-04-07T03:46:16.515-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9830000 QueuedCount 50000 2016-04-07T03:46:16.912-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9840000 QueuedCount 50000 2016-04-07T03:46:17.302-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9850000 QueuedCount 50000 2016-04-07T03:46:17.378-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:17.673-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9860000 QueuedCount 50000 2016-04-07T03:46:18.025-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9870000 QueuedCount 50000 2016-04-07T03:46:18.433-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9880000 QueuedCount 50000 2016-04-07T03:46:18.811-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9890000 QueuedCount 46832 2016-04-07T03:46:19.182-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9900000 QueuedCount 36832 2016-04-07T03:46:19.538-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9910000 QueuedCount 26832 2016-04-07T03:46:19.883-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9920000 QueuedCount 16832 2016-04-07T03:46:20.227-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9930000 QueuedCount 6832 2016-04-07T03:46:20.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 571 2016-04-07T03:46:20.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 571 2016-04-07T03:46:20.929-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.072µs CommitTime 473.672728ms TotalFlushTime 6m12.093805779s TotalCommitTime 21.293169152s 2016-04-07T03:46:20.929-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 9936832, 9936832, 69 committed:true 2016-04-07T03:46:20.929-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 9936832, 9936832, 69 committed:true) SnapCreateDur 484.185349ms SnapOpenDur 44.578µs 2016-04-07T03:46:20.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 571 2016-04-07T03:46:20.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 571 2016-04-07T03:46:20.973-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:21.017-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18060000 2016-04-07T03:46:21.059-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18070000 2016-04-07T03:46:21.062-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9940000 QueuedCount 50000 2016-04-07T03:46:21.104-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18080000 2016-04-07T03:46:21.152-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18090000 2016-04-07T03:46:21.492-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9950000 QueuedCount 50000 2016-04-07T03:46:21.819-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:21.888-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9960000 QueuedCount 50000 2016-04-07T03:46:22.301-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9970000 QueuedCount 50000 2016-04-07T03:46:22.71-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9980000 QueuedCount 50000 2016-04-07T03:46:23.098-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 9990000 QueuedCount 50000 2016-04-07T03:46:23.253-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:23.388-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18100000 2016-04-07T03:46:23.51-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10000000 QueuedCount 50000 2016-04-07T03:46:23.905-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10010000 QueuedCount 50000 2016-04-07T03:46:24.298-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10020000 QueuedCount 50000 2016-04-07T03:46:24.682-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10030000 QueuedCount 48766 2016-04-07T03:46:25.033-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10040000 QueuedCount 38766 2016-04-07T03:46:25.398-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10050000 QueuedCount 28766 2016-04-07T03:46:27.328-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10060000 QueuedCount 18766 2016-04-07T03:46:27.68-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10070000 QueuedCount 8766 2016-04-07T03:46:28.425-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 575 2016-04-07T03:46:28.432-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.377µs CommitTime 445.629963ms TotalFlushTime 6m18.894363504s TotalCommitTime 21.738799115s 2016-04-07T03:46:28.432-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 10078766, 10078766, 70 committed:true 2016-04-07T03:46:28.432-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 10078766, 10078766, 70 committed:true) SnapCreateDur 456.12916ms SnapOpenDur 43.018µs 2016-04-07T03:46:28.474-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10080000 QueuedCount 33595 2016-04-07T03:46:28.478-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18110000 2016-04-07T03:46:28.527-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18120000 2016-04-07T03:46:28.897-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10090000 QueuedCount 25626 2016-04-07T03:46:29.302-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10100000 QueuedCount 15626 2016-04-07T03:46:29.668-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10110000 QueuedCount 5626 2016-04-07T03:46:30.159-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.604µs CommitTime 308.914183ms TotalFlushTime 6m20.247536062s TotalCommitTime 22.047713298s 2016-04-07T03:46:30.159-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 10115626, 10115626, 71 committed:true 2016-04-07T03:46:30.159-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 10115626, 10115626, 71 committed:true) SnapCreateDur 321.935749ms SnapOpenDur 36.733µs 2016-04-07T03:46:30.204-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:30.226-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18130000 2016-04-07T03:46:30.577-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18140000 2016-04-07T03:46:30.638-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18150000 2016-04-07T03:46:30.644-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10120000 QueuedCount 50000 2016-04-07T03:46:30.699-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18160000 2016-04-07T03:46:31.071-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10130000 QueuedCount 50000 2016-04-07T03:46:31.262-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:31.422-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10140000 QueuedCount 50000 2016-04-07T03:46:31.813-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10150000 QueuedCount 50000 2016-04-07T03:46:32.206-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10160000 QueuedCount 50000 2016-04-07T03:46:32.601-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10170000 QueuedCount 50000 2016-04-07T03:46:32.686-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:32.986-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10180000 QueuedCount 50000 2016-04-07T03:46:33.387-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10190000 QueuedCount 50000 2016-04-07T03:46:33.788-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10200000 QueuedCount 50000 2016-04-07T03:46:34.104-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:34.169-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10210000 QueuedCount 50000 2016-04-07T03:46:34.486-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10220000 QueuedCount 50000 2016-04-07T03:46:34.914-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10230000 QueuedCount 50000 2016-04-07T03:46:35.308-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10240000 QueuedCount 50000 2016-04-07T03:46:35.679-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10250000 QueuedCount 45592 2016-04-07T03:46:35.762-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 585 2016-04-07T03:46:35.792-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 585 2016-04-07T03:46:35.822-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 585 2016-04-07T03:46:35.852-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 585 2016-04-07T03:46:35.882-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 585 2016-04-07T03:46:35.912-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 585 2016-04-07T03:46:35.942-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 585 2016-04-07T03:46:35.972-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 585 2016-04-07T03:46:36.002-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 585 2016-04-07T03:46:36.011-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10260000 QueuedCount 35592 2016-04-07T03:46:36.032-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 585 2016-04-07T03:46:36.062-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 585 2016-04-07T03:46:36.092-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 585 2016-04-07T03:46:36.122-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 585 2016-04-07T03:46:36.152-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 585 2016-04-07T03:46:36.182-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 585 2016-04-07T03:46:36.212-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 585 2016-04-07T03:46:36.242-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 585 2016-04-07T03:46:36.272-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 585 2016-04-07T03:46:36.302-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 585 2016-04-07T03:46:36.332-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 585 2016-04-07T03:46:36.351-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10270000 QueuedCount 25592 2016-04-07T03:46:36.362-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 585 2016-04-07T03:46:36.392-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 585 2016-04-07T03:46:36.422-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 585 2016-04-07T03:46:36.452-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 585 2016-04-07T03:46:36.482-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 585 2016-04-07T03:46:36.512-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 585 2016-04-07T03:46:36.542-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 585 2016-04-07T03:46:36.572-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 585 2016-04-07T03:46:36.602-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 585 2016-04-07T03:46:36.632-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 585 2016-04-07T03:46:36.662-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 585 2016-04-07T03:46:36.688-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10280000 QueuedCount 15592 2016-04-07T03:46:36.692-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 585 2016-04-07T03:46:36.722-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 585 2016-04-07T03:46:36.752-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 585 2016-04-07T03:46:36.782-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 585 2016-04-07T03:46:36.812-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 585 2016-04-07T03:46:36.842-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 585 2016-04-07T03:46:36.872-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 585 2016-04-07T03:46:36.902-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 585 2016-04-07T03:46:36.932-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 585 2016-04-07T03:46:36.962-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 585 2016-04-07T03:46:36.992-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 585 2016-04-07T03:46:37.022-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 585 2016-04-07T03:46:37.034-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10290000 QueuedCount 5592 2016-04-07T03:46:37.052-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 585 2016-04-07T03:46:37.082-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 585 2016-04-07T03:46:37.112-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 585 2016-04-07T03:46:37.142-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 585 2016-04-07T03:46:37.172-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 585 2016-04-07T03:46:37.202-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 585 2016-04-07T03:46:37.232-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 585 2016-04-07T03:46:37.262-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 585 2016-04-07T03:46:37.292-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 585 2016-04-07T03:46:37.322-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 585 2016-04-07T03:46:37.352-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 585 2016-04-07T03:46:37.382-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 585 2016-04-07T03:46:37.412-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 585 2016-04-07T03:46:37.442-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 585 2016-04-07T03:46:37.472-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 585 2016-04-07T03:46:37.502-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 585 2016-04-07T03:46:37.532-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 585 2016-04-07T03:46:37.562-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 585 2016-04-07T03:46:37.592-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 585 2016-04-07T03:46:37.622-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 585 2016-04-07T03:46:37.652-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 585 2016-04-07T03:46:37.666-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.827µs CommitTime 457.346455ms TotalFlushTime 6m26.979339862s TotalCommitTime 22.505059753s 2016-04-07T03:46:37.666-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 10295592, 10295592, 72 committed:true 2016-04-07T03:46:37.666-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 10295592, 10295592, 72 committed:true) SnapCreateDur 467.838262ms SnapOpenDur 36.403µs 2016-04-07T03:46:37.682-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 585 2016-04-07T03:46:37.71-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:37.717-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18170000 2016-04-07T03:46:37.753-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18180000 2016-04-07T03:46:37.799-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18190000 2016-04-07T03:46:37.839-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18200000 2016-04-07T03:46:37.888-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18210000 2016-04-07T03:46:37.889-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10300000 QueuedCount 50000 2016-04-07T03:46:37.928-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18220000 2016-04-07T03:46:38.339-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10310000 QueuedCount 50000 2016-04-07T03:46:38.604-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:38.691-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10320000 QueuedCount 50000 2016-04-07T03:46:39.199-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10330000 QueuedCount 50000 2016-04-07T03:46:39.604-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10340000 QueuedCount 50000 2016-04-07T03:46:40.018-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10350000 QueuedCount 50000 2016-04-07T03:46:40.1-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:40.281-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18230000 2016-04-07T03:46:40.421-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10360000 QueuedCount 50000 2016-04-07T03:46:40.591-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18240000 2016-04-07T03:46:40.796-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10370000 QueuedCount 50000 2016-04-07T03:46:40.808-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18250000 2016-04-07T03:46:41.058-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18260000 2016-04-07T03:46:41.16-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18270000 2016-04-07T03:46:41.186-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10380000 QueuedCount 50000 2016-04-07T03:46:41.636-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10390000 QueuedCount 44138 2016-04-07T03:46:43.39-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10400000 QueuedCount 34138 2016-04-07T03:46:43.71-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10410000 QueuedCount 24138 2016-04-07T03:46:44.045-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10420000 QueuedCount 14138 2016-04-07T03:46:44.384-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10430000 QueuedCount 4138 2016-04-07T03:46:44.968-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 12.307µs CommitTime 438.816169ms TotalFlushTime 6m33.598106047s TotalCommitTime 22.943875922s 2016-04-07T03:46:44.969-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 10434138, 10434138, 73 committed:true 2016-04-07T03:46:44.969-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 10434138, 10434138, 73 committed:true) SnapCreateDur 449.21779ms SnapOpenDur 43.313µs 2016-04-07T03:46:45.013-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:45.068-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18280000 2016-04-07T03:46:45.109-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18290000 2016-04-07T03:46:45.15-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18300000 2016-04-07T03:46:45.195-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18310000 2016-04-07T03:46:45.212-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10440000 QueuedCount 50000 2016-04-07T03:46:45.673-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10450000 QueuedCount 50000 2016-04-07T03:46:45.773-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:46.116-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10460000 QueuedCount 50000 2016-04-07T03:46:46.524-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10470000 QueuedCount 50000 2016-04-07T03:46:46.943-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10480000 QueuedCount 50000 2016-04-07T03:46:47.35-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10490000 QueuedCount 46137 2016-04-07T03:46:47.727-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10500000 QueuedCount 36137 2016-04-07T03:46:48.11-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10510000 QueuedCount 26137 2016-04-07T03:46:48.444-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10520000 QueuedCount 16137 2016-04-07T03:46:48.827-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10530000 QueuedCount 6137 2016-04-07T03:46:49.471-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.509µs CommitTime 407.836968ms TotalFlushTime 6m37.531234149s TotalCommitTime 23.35171289s 2016-04-07T03:46:49.471-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 10536137, 10536137, 74 committed:true 2016-04-07T03:46:49.471-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 10536137, 10536137, 74 committed:true) SnapCreateDur 418.442398ms SnapOpenDur 59.347µs 2016-04-07T03:46:49.514-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:49.523-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18320000 2016-04-07T03:46:49.592-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18330000 2016-04-07T03:46:49.614-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10540000 QueuedCount 50000 2016-04-07T03:46:49.626-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18340000 2016-04-07T03:46:49.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18350000 2016-04-07T03:46:50.06-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10550000 QueuedCount 50000 2016-04-07T03:46:50.233-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:50.537-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10560000 QueuedCount 50000 2016-04-07T03:46:50.644-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18360000 2016-04-07T03:46:50.968-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10570000 QueuedCount 50000 2016-04-07T03:46:51.397-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10580000 QueuedCount 50000 2016-04-07T03:46:51.718-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:51.801-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10590000 QueuedCount 50000 2016-04-07T03:46:52.205-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10600000 QueuedCount 50000 2016-04-07T03:46:52.592-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10610000 QueuedCount 50000 2016-04-07T03:46:52.937-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10620000 QueuedCount 50000 2016-04-07T03:46:53.31-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10630000 QueuedCount 40283 2016-04-07T03:46:53.648-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10640000 QueuedCount 30283 2016-04-07T03:46:54.001-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10650000 QueuedCount 20283 2016-04-07T03:46:54.348-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10660000 QueuedCount 10283 2016-04-07T03:46:54.682-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10670000 QueuedCount 283 2016-04-07T03:46:55.117-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.14µs CommitTime 422.223948ms TotalFlushTime 6m42.526247245s TotalCommitTime 23.773936838s 2016-04-07T03:46:55.118-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 10670283, 10670283, 75 committed:true 2016-04-07T03:46:55.118-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 10670283, 10670283, 75 committed:true) SnapCreateDur 432.060169ms SnapOpenDur 42.355µs 2016-04-07T03:46:55.155-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18370000 2016-04-07T03:46:55.156-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:55.186-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18380000 2016-04-07T03:46:55.226-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18390000 2016-04-07T03:46:55.26-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18400000 2016-04-07T03:46:55.565-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10680000 QueuedCount 50000 2016-04-07T03:46:55.956-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:56.049-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10690000 QueuedCount 50000 2016-04-07T03:46:56.225-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18410000 2016-04-07T03:46:56.476-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18420000 2016-04-07T03:46:56.48-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10700000 QueuedCount 50000 2016-04-07T03:46:56.894-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10710000 QueuedCount 50000 2016-04-07T03:46:57.342-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10720000 QueuedCount 50000 2016-04-07T03:46:57.526-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:46:57.783-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10730000 QueuedCount 50000 2016-04-07T03:46:57.799-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18430000 2016-04-07T03:46:58.209-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10740000 QueuedCount 50000 2016-04-07T03:46:58.574-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10750000 QueuedCount 50000 2016-04-07T03:46:58.99-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10760000 QueuedCount 49479 2016-04-07T03:46:59.376-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10770000 QueuedCount 39479 2016-04-07T03:46:59.742-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10780000 QueuedCount 29479 2016-04-07T03:47:00.109-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10790000 QueuedCount 19479 2016-04-07T03:47:00.481-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10800000 QueuedCount 9479 2016-04-07T03:47:01.239-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.272µs CommitTime 454.811766ms TotalFlushTime 6m47.950457378s TotalCommitTime 24.228748604s 2016-04-07T03:47:01.239-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 10809479, 10809479, 76 committed:true 2016-04-07T03:47:01.239-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 10809479, 10809479, 76 committed:true) SnapCreateDur 464.735706ms SnapOpenDur 32.419µs 2016-04-07T03:47:01.272-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10810000 QueuedCount 23936 2016-04-07T03:47:01.279-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:01.314-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18440000 2016-04-07T03:47:01.773-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10820000 QueuedCount 50000 2016-04-07T03:47:01.981-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:02.224-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10830000 QueuedCount 50000 2016-04-07T03:47:02.251-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18450000 2016-04-07T03:47:02.438-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18460000 2016-04-07T03:47:02.483-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18470000 2016-04-07T03:47:02.595-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10840000 QueuedCount 50000 2016-04-07T03:47:03.018-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10850000 QueuedCount 50000 2016-04-07T03:47:03.429-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:03.479-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18480000 2016-04-07T03:47:03.512-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10860000 QueuedCount 50000 2016-04-07T03:47:03.88-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18490000 2016-04-07T03:47:03.959-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10870000 QueuedCount 50000 2016-04-07T03:47:04.072-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18500000 2016-04-07T03:47:04.355-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10880000 QueuedCount 50000 2016-04-07T03:47:04.75-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10890000 QueuedCount 50000 2016-04-07T03:47:05.132-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10900000 QueuedCount 40525 2016-04-07T03:47:05.503-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10910000 QueuedCount 30525 2016-04-07T03:47:05.829-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10920000 QueuedCount 20525 2016-04-07T03:47:06.228-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10930000 QueuedCount 10525 2016-04-07T03:47:06.59-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10940000 QueuedCount 525 2016-04-07T03:47:07.066-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.814µs CommitTime 457.622833ms TotalFlushTime 6m53.081110599s TotalCommitTime 24.686371437s 2016-04-07T03:47:07.066-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 10940525, 10940525, 77 committed:true 2016-04-07T03:47:07.066-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 10940525, 10940525, 77 committed:true) SnapCreateDur 468.611038ms SnapOpenDur 40.685µs 2016-04-07T03:47:07.105-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:07.117-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18510000 2016-04-07T03:47:07.149-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18520000 2016-04-07T03:47:07.186-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18530000 2016-04-07T03:47:07.488-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10950000 QueuedCount 50000 2016-04-07T03:47:07.668-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:07.79-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18540000 2016-04-07T03:47:07.921-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10960000 QueuedCount 50000 2016-04-07T03:47:08.011-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18550000 2016-04-07T03:47:08.334-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10970000 QueuedCount 50000 2016-04-07T03:47:08.739-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10980000 QueuedCount 50000 2016-04-07T03:47:09.035-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:09.209-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 10990000 QueuedCount 50000 2016-04-07T03:47:09.221-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18560000 2016-04-07T03:47:10.439-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11000000 QueuedCount 50000 2016-04-07T03:47:11.833-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11010000 QueuedCount 50000 2016-04-07T03:47:12.221-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11020000 QueuedCount 49535 2016-04-07T03:47:12.576-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11030000 QueuedCount 39535 2016-04-07T03:47:12.907-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11040000 QueuedCount 29535 2016-04-07T03:47:12.943-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":7494403,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":5843178500,"bucket-1:myindex:avg_ts_items_count":131972,"bucket-1:myindex:build_progress":55,"bucket-1:myindex:data_size":3536975577,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":17167839232,"bucket-1:myindex:flush_queue_size":28634,"bucket-1:myindex:frag_percent":79,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":1153702200,"bucket-1:myindex:items_count":10940525,"bucket-1:myindex:num_commits":77,"bucket-1:myindex:num_compactions":2,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":11036952,"bucket-1:myindex:num_docs_pending":1444119,"bucket-1:myindex:num_docs_queued":7518929,"bucket-1:myindex:num_flush_queued":11069535,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":77,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":5826830947,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"77 24686371437 9035922919589360107","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"11040902 69752947070 610664110098918368","bucket-1:myindex:timings/storage_handle_open":"77 1818552 44570649906","bucket-1:myindex:timings/storage_info":"309 2955266 73966048792","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"78 62444714 727356194028416","bucket-1:myindex:timings/storage_meta_set":"77 357451633 107826705219343027","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"22081802 341825005247 -5702014937943735135","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":18563938,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":491,"memory_quota":31457280000,"memory_used":15345471488,"needs_restart":false,"num_connections":0,"timings/stats_response":"2057 233407875 38783014959173"} 2016-04-07T03:47:13.244-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11050000 QueuedCount 19535 2016-04-07T03:47:13.61-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11060000 QueuedCount 9535 2016-04-07T03:47:14.422-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.381µs CommitTime 484.934661ms TotalFlushTime 6m59.720868956s TotalCommitTime 25.171306098s 2016-04-07T03:47:14.422-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 11069535, 11069535, 78 committed:true 2016-04-07T03:47:14.422-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 11069535, 11069535, 78 committed:true) SnapCreateDur 495.965072ms SnapOpenDur 32.857µs 2016-04-07T03:47:14.426-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 652 2016-04-07T03:47:14.455-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11070000 QueuedCount 23360 2016-04-07T03:47:14.478-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18570000 2016-04-07T03:47:14.515-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18580000 2016-04-07T03:47:15.947-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11080000 QueuedCount 17422 2016-04-07T03:47:17.1-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11090000 QueuedCount 7422 2016-04-07T03:47:17.654-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.952µs CommitTime 255.197383ms TotalFlushTime 7m2.64602323s TotalCommitTime 25.426503481s 2016-04-07T03:47:17.654-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 11097422, 11097422, 79 committed:true 2016-04-07T03:47:17.654-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 11097422, 11097422, 79 committed:true) SnapCreateDur 265.012085ms SnapOpenDur 45.911µs 2016-04-07T03:47:17.691-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:17.721-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:17.786-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11100000 QueuedCount 50000 2016-04-07T03:47:17.788-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:17.793-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18590000 2016-04-07T03:47:17.892-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18600000 2016-04-07T03:47:18.057-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:18.134-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:18.2-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11110000 QueuedCount 50000 2016-04-07T03:47:18.22-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:18.493-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:18.618-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18610000 2016-04-07T03:47:18.679-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11120000 QueuedCount 50000 2016-04-07T03:47:18.685-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:18.807-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18620000 2016-04-07T03:47:18.954-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:20.374-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:20.687-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11130000 QueuedCount 50000 2016-04-07T03:47:21.335-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11140000 QueuedCount 45291 2016-04-07T03:47:21.757-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11150000 QueuedCount 35291 2016-04-07T03:47:22.178-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11160000 QueuedCount 25291 2016-04-07T03:47:22.596-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11170000 QueuedCount 15291 2016-04-07T03:47:23.019-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11180000 QueuedCount 5291 2016-04-07T03:47:23.81-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.378µs CommitTime 593.900777ms TotalFlushTime 7m8.078166094s TotalCommitTime 26.020404258s 2016-04-07T03:47:23.81-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 11185291, 11185291, 80 committed:true 2016-04-07T03:47:23.81-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 11185291, 11185291, 80 committed:true) SnapCreateDur 605.135908ms SnapOpenDur 41.209µs 2016-04-07T03:47:23.833-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:23.854-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:23.875-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:23.899-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:23.92-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18630000 2016-04-07T03:47:23.921-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:23.945-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:23.967-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:23.994-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18640000 2016-04-07T03:47:23.995-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:24.029-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18650000 2016-04-07T03:47:24.045-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11190000 QueuedCount 50000 2016-04-07T03:47:24.064-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18660000 2016-04-07T03:47:24.282-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18670000 2016-04-07T03:47:24.32-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18680000 2016-04-07T03:47:24.555-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11200000 QueuedCount 50000 2016-04-07T03:47:26.134-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11210000 QueuedCount 50000 2016-04-07T03:47:27.208-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11220000 QueuedCount 50000 2016-04-07T03:47:27.622-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:27.625-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11230000 QueuedCount 49738 2016-04-07T03:47:28.038-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11240000 QueuedCount 50000 2016-04-07T03:47:28.393-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11250000 QueuedCount 40919 2016-04-07T03:47:28.735-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11260000 QueuedCount 30919 2016-04-07T03:47:29.097-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11270000 QueuedCount 20919 2016-04-07T03:47:29.449-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11280000 QueuedCount 10919 2016-04-07T03:47:29.886-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11290000 QueuedCount 919 2016-04-07T03:47:30.335-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.726µs CommitTime 420.531429ms TotalFlushTime 7m14.010670698s TotalCommitTime 26.440935687s 2016-04-07T03:47:30.335-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 11290919, 11290919, 81 committed:true 2016-04-07T03:47:30.335-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 11290919, 11290919, 81 committed:true) SnapCreateDur 431.42184ms SnapOpenDur 39.032µs 2016-04-07T03:47:30.357-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.378-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.4-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.422-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.444-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.466-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.489-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.512-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.535-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.557-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.581-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.602-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:30.633-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18690000 2016-04-07T03:47:30.667-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18700000 2016-04-07T03:47:30.757-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11300000 QueuedCount 50000 2016-04-07T03:47:31.206-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11310000 QueuedCount 50000 2016-04-07T03:47:31.224-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:31.633-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18710000 2016-04-07T03:47:31.7-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11320000 QueuedCount 50000 2016-04-07T03:47:33.773-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18720000 2016-04-07T03:47:33.85-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11330000 QueuedCount 50000 2016-04-07T03:47:34.226-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11340000 QueuedCount 50000 2016-04-07T03:47:34.234-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18730000 2016-04-07T03:47:34.275-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18740000 2016-04-07T03:47:34.649-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11350000 QueuedCount 50000 2016-04-07T03:47:35.045-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:35.049-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11360000 QueuedCount 49638 2016-04-07T03:47:35.451-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18750000 2016-04-07T03:47:35.457-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11370000 QueuedCount 50000 2016-04-07T03:47:35.859-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11380000 QueuedCount 50000 2016-04-07T03:47:36.252-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11390000 QueuedCount 50000 2016-04-07T03:47:36.648-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11400000 QueuedCount 50000 2016-04-07T03:47:37.007-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11410000 QueuedCount 46061 2016-04-07T03:47:37.349-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11420000 QueuedCount 36061 2016-04-07T03:47:37.644-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11430000 QueuedCount 26061 2016-04-07T03:47:38.03-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11440000 QueuedCount 16061 2016-04-07T03:47:38.368-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11450000 QueuedCount 6061 2016-04-07T03:47:39.019-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.614µs CommitTime 445.17289ms TotalFlushTime 7m21.965402469s TotalCommitTime 26.886108577s 2016-04-07T03:47:39.019-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 11456061, 11456061, 82 committed:true 2016-04-07T03:47:39.019-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 11456061, 11456061, 82 committed:true) SnapCreateDur 454.984279ms SnapOpenDur 41.953µs 2016-04-07T03:47:39.068-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:39.089-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18760000 2016-04-07T03:47:39.136-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18770000 2016-04-07T03:47:39.18-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11460000 QueuedCount 50000 2016-04-07T03:47:39.273-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18780000 2016-04-07T03:47:39.494-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18790000 2016-04-07T03:47:39.656-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11470000 QueuedCount 50000 2016-04-07T03:47:40.137-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11480000 QueuedCount 50000 2016-04-07T03:47:40.612-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11490000 QueuedCount 50000 2016-04-07T03:47:41.05-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11500000 QueuedCount 50000 2016-04-07T03:47:41.063-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:41.254-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18800000 2016-04-07T03:47:41.52-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11510000 QueuedCount 50000 2016-04-07T03:47:41.992-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11520000 QueuedCount 50000 2016-04-07T03:47:42.403-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11530000 QueuedCount 50000 2016-04-07T03:47:42.746-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11540000 QueuedCount 50000 2016-04-07T03:47:43.134-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11550000 QueuedCount 46762 2016-04-07T03:47:43.556-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11560000 QueuedCount 36762 2016-04-07T03:47:43.97-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11570000 QueuedCount 26762 2016-04-07T03:47:44.335-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11580000 QueuedCount 16762 2016-04-07T03:47:44.688-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11590000 QueuedCount 6762 2016-04-07T03:47:45.397-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.565µs CommitTime 473.207779ms TotalFlushTime 7m27.593726075s TotalCommitTime 27.359316356s 2016-04-07T03:47:45.397-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 11596762, 11596762, 83 committed:true 2016-04-07T03:47:45.397-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 11596762, 11596762, 83 committed:true) SnapCreateDur 484.030139ms SnapOpenDur 41.172µs 2016-04-07T03:47:45.437-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18810000 2016-04-07T03:47:45.448-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:45.541-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11600000 QueuedCount 50000 2016-04-07T03:47:45.832-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18820000 2016-04-07T03:47:46.022-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11610000 QueuedCount 50000 2016-04-07T03:47:46.201-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18830000 2016-04-07T03:47:46.483-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11620000 QueuedCount 50000 2016-04-07T03:47:46.602-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18840000 2016-04-07T03:47:46.831-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18850000 2016-04-07T03:47:46.954-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11630000 QueuedCount 50000 2016-04-07T03:47:47.343-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11640000 QueuedCount 50000 2016-04-07T03:47:47.362-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:47.541-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18860000 2016-04-07T03:47:47.743-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11650000 QueuedCount 50000 2016-04-07T03:47:48.186-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11660000 QueuedCount 50000 2016-04-07T03:47:48.588-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11670000 QueuedCount 50000 2016-04-07T03:47:48.986-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11680000 QueuedCount 49995 2016-04-07T03:47:49.447-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11690000 QueuedCount 50000 2016-04-07T03:47:49.81-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11700000 QueuedCount 40067 2016-04-07T03:47:50.164-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11710000 QueuedCount 30067 2016-04-07T03:47:50.537-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11720000 QueuedCount 20067 2016-04-07T03:47:50.898-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11730000 QueuedCount 10067 2016-04-07T03:47:51.183-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11740000 QueuedCount 67 2016-04-07T03:47:51.671-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.219µs CommitTime 473.272353ms TotalFlushTime 7m33.122135442s TotalCommitTime 27.832588709s 2016-04-07T03:47:51.671-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 11740067, 11740067, 84 committed:true 2016-04-07T03:47:51.671-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 11740067, 11740067, 84 committed:true) SnapCreateDur 483.073877ms SnapOpenDur 43.595µs 2016-04-07T03:47:51.72-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18870000 2016-04-07T03:47:51.723-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:51.759-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18880000 2016-04-07T03:47:51.798-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18890000 2016-04-07T03:47:51.921-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18900000 2016-04-07T03:47:52.098-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11750000 QueuedCount 50000 2016-04-07T03:47:52.544-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11760000 QueuedCount 50000 2016-04-07T03:47:52.997-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11770000 QueuedCount 50000 2016-04-07T03:47:53.446-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11780000 QueuedCount 50000 2016-04-07T03:47:53.685-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:53.732-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18910000 2016-04-07T03:47:53.863-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11790000 QueuedCount 50000 2016-04-07T03:47:53.869-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18920000 2016-04-07T03:47:54.268-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11800000 QueuedCount 50000 2016-04-07T03:47:54.297-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18930000 2016-04-07T03:47:54.551-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18940000 2016-04-07T03:47:54.599-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18950000 2016-04-07T03:47:54.693-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11810000 QueuedCount 50000 2016-04-07T03:47:55.092-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11820000 QueuedCount 50000 2016-04-07T03:47:55.433-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11830000 QueuedCount 50000 2016-04-07T03:47:55.853-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11840000 QueuedCount 47024 2016-04-07T03:47:56.213-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11850000 QueuedCount 37024 2016-04-07T03:47:56.563-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11860000 QueuedCount 27024 2016-04-07T03:47:56.914-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11870000 QueuedCount 17024 2016-04-07T03:47:57.25-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11880000 QueuedCount 7024 2016-04-07T03:47:57.948-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.353µs CommitTime 475.832973ms TotalFlushTime 7m38.664908344s TotalCommitTime 28.308421682s 2016-04-07T03:47:57.948-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 11887024, 11887024, 85 committed:true 2016-04-07T03:47:57.948-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 11887024, 11887024, 85 committed:true) SnapCreateDur 485.735809ms SnapOpenDur 41.544µs 2016-04-07T03:47:58.03-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18960000 2016-04-07T03:47:58.105-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11890000 QueuedCount 50000 2016-04-07T03:47:58.199-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:47:58.434-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18970000 2016-04-07T03:47:58.664-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11900000 QueuedCount 50000 2016-04-07T03:47:59.018-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18980000 2016-04-07T03:48:00.631-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11910000 QueuedCount 50000 2016-04-07T03:48:01.577-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11920000 QueuedCount 50000 2016-04-07T03:48:01.959-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11930000 QueuedCount 50000 2016-04-07T03:48:02.434-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11940000 QueuedCount 50000 2016-04-07T03:48:02.508-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:02.814-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 18990000 2016-04-07T03:48:02.934-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11950000 QueuedCount 50000 2016-04-07T03:48:03.432-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11960000 QueuedCount 50000 2016-04-07T03:48:03.849-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11970000 QueuedCount 50000 2016-04-07T03:48:04.365-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11980000 QueuedCount 50000 2016-04-07T03:48:06.663-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 11990000 QueuedCount 48094 2016-04-07T03:48:07.856-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 734 2016-04-07T03:48:07.886-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 734 2016-04-07T03:48:07.916-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 734 2016-04-07T03:48:07.946-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 734 2016-04-07T03:48:07.976-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 734 2016-04-07T03:48:08.006-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 734 2016-04-07T03:48:08.036-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 734 2016-04-07T03:48:08.066-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 734 2016-04-07T03:48:08.096-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 734 2016-04-07T03:48:08.126-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 734 2016-04-07T03:48:08.156-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 734 2016-04-07T03:48:08.186-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 734 2016-04-07T03:48:08.216-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 734 2016-04-07T03:48:08.246-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 734 2016-04-07T03:48:08.276-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 734 2016-04-07T03:48:08.29-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12000000 QueuedCount 38094 2016-04-07T03:48:08.306-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 734 2016-04-07T03:48:08.336-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 734 2016-04-07T03:48:08.366-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 734 2016-04-07T03:48:08.396-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 734 2016-04-07T03:48:08.426-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 734 2016-04-07T03:48:08.456-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 734 2016-04-07T03:48:08.486-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 734 2016-04-07T03:48:08.516-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 734 2016-04-07T03:48:08.546-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 734 2016-04-07T03:48:08.576-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 734 2016-04-07T03:48:08.606-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 734 2016-04-07T03:48:08.636-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 734 2016-04-07T03:48:08.666-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 734 2016-04-07T03:48:08.696-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 734 2016-04-07T03:48:08.726-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 734 2016-04-07T03:48:08.756-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 734 2016-04-07T03:48:08.786-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 734 2016-04-07T03:48:08.816-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 734 2016-04-07T03:48:08.846-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 734 2016-04-07T03:48:08.876-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 734 2016-04-07T03:48:08.906-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 734 2016-04-07T03:48:08.936-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 734 2016-04-07T03:48:08.966-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 734 2016-04-07T03:48:08.996-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 734 2016-04-07T03:48:09.026-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 734 2016-04-07T03:48:09.056-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 734 2016-04-07T03:48:09.086-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 734 2016-04-07T03:48:09.116-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 734 2016-04-07T03:48:09.146-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 734 2016-04-07T03:48:09.169-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12010000 QueuedCount 28094 2016-04-07T03:48:09.176-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 734 2016-04-07T03:48:09.206-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 734 2016-04-07T03:48:09.236-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 734 2016-04-07T03:48:09.266-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 734 2016-04-07T03:48:09.296-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 734 2016-04-07T03:48:09.326-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 734 2016-04-07T03:48:09.356-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 734 2016-04-07T03:48:09.386-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 734 2016-04-07T03:48:09.416-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 734 2016-04-07T03:48:09.446-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 734 2016-04-07T03:48:09.476-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 734 2016-04-07T03:48:09.506-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 734 2016-04-07T03:48:09.523-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12020000 QueuedCount 18094 2016-04-07T03:48:09.536-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 734 2016-04-07T03:48:09.566-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 734 2016-04-07T03:48:09.596-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 734 2016-04-07T03:48:09.626-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 734 2016-04-07T03:48:09.656-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 734 2016-04-07T03:48:09.686-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 734 2016-04-07T03:48:09.716-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 734 2016-04-07T03:48:09.746-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 734 2016-04-07T03:48:09.776-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 734 2016-04-07T03:48:09.806-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 734 2016-04-07T03:48:09.836-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 734 2016-04-07T03:48:09.866-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 734 2016-04-07T03:48:09.896-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 734 2016-04-07T03:48:09.926-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 734 2016-04-07T03:48:09.929-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12030000 QueuedCount 8094 2016-04-07T03:48:09.956-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 734 2016-04-07T03:48:09.986-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 734 2016-04-07T03:48:10.016-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 734 2016-04-07T03:48:10.046-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 734 2016-04-07T03:48:10.076-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 734 2016-04-07T03:48:10.106-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 734 2016-04-07T03:48:10.136-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 734 2016-04-07T03:48:10.166-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 734 2016-04-07T03:48:10.196-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 734 2016-04-07T03:48:10.226-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 734 2016-04-07T03:48:10.256-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 734 2016-04-07T03:48:10.286-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 734 2016-04-07T03:48:10.316-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 734 2016-04-07T03:48:10.346-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 734 2016-04-07T03:48:10.376-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 734 2016-04-07T03:48:10.406-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 734 2016-04-07T03:48:10.436-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 734 2016-04-07T03:48:10.466-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 734 2016-04-07T03:48:10.496-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 734 2016-04-07T03:48:10.526-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 734 2016-04-07T03:48:10.556-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 734 2016-04-07T03:48:10.586-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 734 2016-04-07T03:48:10.616-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 734 2016-04-07T03:48:10.646-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 734 2016-04-07T03:48:10.676-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 734 2016-04-07T03:48:10.706-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 734 2016-04-07T03:48:10.736-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 734 2016-04-07T03:48:10.766-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 734 2016-04-07T03:48:10.796-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 734 2016-04-07T03:48:10.81-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.54µs CommitTime 545.832603ms TotalFlushTime 7m50.663971154s TotalCommitTime 28.854254285s 2016-04-07T03:48:10.81-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 12038094, 12038094, 86 committed:true 2016-04-07T03:48:10.81-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 12038094, 12038094, 86 committed:true) SnapCreateDur 557.546408ms SnapOpenDur 45.689µs 2016-04-07T03:48:10.826-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 734 2016-04-07T03:48:10.875-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12040000 QueuedCount 48635 2016-04-07T03:48:10.904-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19000000 2016-04-07T03:48:10.957-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19010000 2016-04-07T03:48:11.01-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19020000 2016-04-07T03:48:11.41-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12050000 QueuedCount 38635 2016-04-07T03:48:11.88-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12060000 QueuedCount 28635 2016-04-07T03:48:12.339-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12070000 QueuedCount 18635 2016-04-07T03:48:12.762-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12080000 QueuedCount 8635 2016-04-07T03:48:12.943-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":6935278,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":9643050882,"bucket-1:myindex:avg_ts_items_count":147097,"bucket-1:myindex:build_progress":59,"bucket-1:myindex:data_size":3873147866,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":19611435008,"bucket-1:myindex:flush_queue_size":3426,"bucket-1:myindex:frag_percent":80,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":1262759925,"bucket-1:myindex:items_count":11887024,"bucket-1:myindex:num_commits":86,"bucket-1:myindex:num_compactions":2,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":11990526,"bucket-1:myindex:num_docs_pending":1009624,"bucket-1:myindex:num_docs_queued":6999850,"bucket-1:myindex:num_flush_queued":12088635,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":86,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":12862163750,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"86 28854254285 -7410527535789976241","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"12085210 76875664226 610717077752670084","bucket-1:myindex:timings/storage_handle_open":"86 2051766 50751341920","bucket-1:myindex:timings/storage_info":"345 3254696 78773129170","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"87 65282702 728270333059350","bucket-1:myindex:timings/storage_meta_set":"86 361117937 107828220316605963","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"24170418 388026818588 -3968855650570984596","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":19023913,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":516,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"2199 257853439 43107858177979"} 2016-04-07T03:48:13.474-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.635µs CommitTime 366.569875ms TotalFlushTime 7m52.896563239s TotalCommitTime 29.22082416s 2016-04-07T03:48:13.474-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 12088635, 12088635, 87 committed:true 2016-04-07T03:48:13.474-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 12088635, 12088635, 87 committed:true) SnapCreateDur 376.53356ms SnapOpenDur 62.075µs 2016-04-07T03:48:13.52-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12090000 QueuedCount 47233 2016-04-07T03:48:13.544-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19030000 2016-04-07T03:48:13.588-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19040000 2016-04-07T03:48:13.631-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19050000 2016-04-07T03:48:13.68-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19060000 2016-04-07T03:48:13.727-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19070000 2016-04-07T03:48:13.739-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:13.772-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19080000 2016-04-07T03:48:13.819-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19090000 2016-04-07T03:48:13.855-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19100000 2016-04-07T03:48:14.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12100000 QueuedCount 50000 2016-04-07T03:48:14.05-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19110000 2016-04-07T03:48:14.51-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12110000 QueuedCount 50000 2016-04-07T03:48:15.028-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12120000 QueuedCount 50000 2016-04-07T03:48:15.471-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12130000 QueuedCount 50000 2016-04-07T03:48:15.88-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12140000 QueuedCount 50000 2016-04-07T03:48:16.164-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:16.247-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12150000 QueuedCount 50000 2016-04-07T03:48:16.249-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19120000 2016-04-07T03:48:16.681-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12160000 QueuedCount 50000 2016-04-07T03:48:16.712-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19130000 2016-04-07T03:48:16.971-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19140000 2016-04-07T03:48:17.112-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12170000 QueuedCount 50000 2016-04-07T03:48:17.558-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12180000 QueuedCount 50000 2016-04-07T03:48:17.978-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12190000 QueuedCount 50000 2016-04-07T03:48:18.377-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12200000 QueuedCount 50000 2016-04-07T03:48:18.762-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12210000 QueuedCount 43172 2016-04-07T03:48:19.17-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12220000 QueuedCount 33172 2016-04-07T03:48:19.637-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12230000 QueuedCount 23172 2016-04-07T03:48:19.984-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12240000 QueuedCount 13172 2016-04-07T03:48:20.373-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12250000 QueuedCount 3172 2016-04-07T03:48:21.052-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.6µs CommitTime 526.485315ms TotalFlushTime 7m59.63915633s TotalCommitTime 29.747309475s 2016-04-07T03:48:21.052-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 12253172, 12253172, 88 committed:true 2016-04-07T03:48:21.052-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 12253172, 12253172, 88 committed:true) SnapCreateDur 537.514471ms SnapOpenDur 37.031µs 2016-04-07T03:48:21.122-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19150000 2016-04-07T03:48:21.165-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19160000 2016-04-07T03:48:21.33-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:21.423-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12260000 QueuedCount 50000 2016-04-07T03:48:21.827-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19170000 2016-04-07T03:48:21.912-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12270000 QueuedCount 50000 2016-04-07T03:48:22.042-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19180000 2016-04-07T03:48:22.082-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19190000 2016-04-07T03:48:22.116-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19200000 2016-04-07T03:48:22.38-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12280000 QueuedCount 50000 2016-04-07T03:48:22.777-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12290000 QueuedCount 50000 2016-04-07T03:48:23.213-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12300000 QueuedCount 50000 2016-04-07T03:48:23.62-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12310000 QueuedCount 50000 2016-04-07T03:48:24.032-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12320000 QueuedCount 44726 2016-04-07T03:48:24.425-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12330000 QueuedCount 34726 2016-04-07T03:48:24.773-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12340000 QueuedCount 24726 2016-04-07T03:48:25.127-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12350000 QueuedCount 14726 2016-04-07T03:48:25.49-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12360000 QueuedCount 4726 2016-04-07T03:48:26.16-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.354µs CommitTime 517.838445ms TotalFlushTime 8m4.048867355s TotalCommitTime 30.26514792s 2016-04-07T03:48:26.16-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 12364726, 12364726, 89 committed:true 2016-04-07T03:48:26.16-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 12364726, 12364726, 89 committed:true) SnapCreateDur 528.075371ms SnapOpenDur 43.739µs 2016-04-07T03:48:26.201-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19210000 2016-04-07T03:48:26.238-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19220000 2016-04-07T03:48:26.284-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19230000 2016-04-07T03:48:26.446-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:26.453-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12370000 QueuedCount 49486 2016-04-07T03:48:26.569-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19240000 2016-04-07T03:48:26.702-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19250000 2016-04-07T03:48:26.747-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19260000 2016-04-07T03:48:26.789-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19270000 2016-04-07T03:48:26.834-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19280000 2016-04-07T03:48:26.878-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19290000 2016-04-07T03:48:26.926-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19300000 2016-04-07T03:48:26.927-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12380000 QueuedCount 50000 2016-04-07T03:48:27.344-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12390000 QueuedCount 50000 2016-04-07T03:48:27.763-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12400000 QueuedCount 50000 2016-04-07T03:48:28.2-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12410000 QueuedCount 50000 2016-04-07T03:48:28.612-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12420000 QueuedCount 50000 2016-04-07T03:48:29.083-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12430000 QueuedCount 42638 2016-04-07T03:48:29.514-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12440000 QueuedCount 32638 2016-04-07T03:48:29.947-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12450000 QueuedCount 22638 2016-04-07T03:48:30.387-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12460000 QueuedCount 12638 2016-04-07T03:48:31.06-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12470000 QueuedCount 2638 2016-04-07T03:48:31.672-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.017µs CommitTime 508.72277ms TotalFlushTime 8m8.876705666s TotalCommitTime 30.77387069s 2016-04-07T03:48:31.672-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 12472638, 12472638, 90 committed:true 2016-04-07T03:48:31.672-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 12472638, 12472638, 90 committed:true) SnapCreateDur 519.874045ms SnapOpenDur 46.127µs 2016-04-07T03:48:32.004-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12480000 QueuedCount 44349 2016-04-07T03:48:32.192-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 558 2016-04-07T03:48:32.222-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 558 2016-04-07T03:48:32.252-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 558 2016-04-07T03:48:32.282-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 558 2016-04-07T03:48:32.312-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 558 2016-04-07T03:48:32.342-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 558 2016-04-07T03:48:32.372-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 558 2016-04-07T03:48:32.402-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 558 2016-04-07T03:48:32.432-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 558 2016-04-07T03:48:32.462-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 558 2016-04-07T03:48:32.492-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 558 2016-04-07T03:48:32.522-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 558 2016-04-07T03:48:32.531-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12490000 QueuedCount 34349 2016-04-07T03:48:32.552-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 558 2016-04-07T03:48:32.582-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 558 2016-04-07T03:48:32.612-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 558 2016-04-07T03:48:32.642-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 558 2016-04-07T03:48:32.672-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 558 2016-04-07T03:48:32.702-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 558 2016-04-07T03:48:32.732-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 558 2016-04-07T03:48:32.762-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 558 2016-04-07T03:48:32.792-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 558 2016-04-07T03:48:32.822-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 558 2016-04-07T03:48:32.852-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 558 2016-04-07T03:48:32.882-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 558 2016-04-07T03:48:32.912-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 558 2016-04-07T03:48:32.942-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 558 2016-04-07T03:48:32.972-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 558 2016-04-07T03:48:32.995-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12500000 QueuedCount 24349 2016-04-07T03:48:33.002-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 558 2016-04-07T03:48:33.032-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 558 2016-04-07T03:48:33.062-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 558 2016-04-07T03:48:33.092-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 558 2016-04-07T03:48:33.122-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 558 2016-04-07T03:48:33.152-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 558 2016-04-07T03:48:33.182-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 558 2016-04-07T03:48:33.212-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 558 2016-04-07T03:48:33.242-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 558 2016-04-07T03:48:33.272-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 558 2016-04-07T03:48:33.302-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 558 2016-04-07T03:48:33.332-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 558 2016-04-07T03:48:33.362-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 558 2016-04-07T03:48:33.392-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 558 2016-04-07T03:48:33.392-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12510000 QueuedCount 14349 2016-04-07T03:48:33.422-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 558 2016-04-07T03:48:33.452-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 558 2016-04-07T03:48:33.482-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 558 2016-04-07T03:48:33.512-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 558 2016-04-07T03:48:33.542-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 558 2016-04-07T03:48:33.572-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 558 2016-04-07T03:48:33.602-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 558 2016-04-07T03:48:33.632-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 558 2016-04-07T03:48:33.662-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 558 2016-04-07T03:48:33.692-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 558 2016-04-07T03:48:33.722-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 558 2016-04-07T03:48:33.752-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 558 2016-04-07T03:48:33.762-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12520000 QueuedCount 4349 2016-04-07T03:48:33.782-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 558 2016-04-07T03:48:33.812-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 558 2016-04-07T03:48:33.842-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 558 2016-04-07T03:48:33.872-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 558 2016-04-07T03:48:33.902-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 558 2016-04-07T03:48:33.932-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 558 2016-04-07T03:48:33.962-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 558 2016-04-07T03:48:33.992-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 558 2016-04-07T03:48:34.022-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 558 2016-04-07T03:48:34.052-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 558 2016-04-07T03:48:34.082-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 558 2016-04-07T03:48:34.112-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 558 2016-04-07T03:48:34.142-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 558 2016-04-07T03:48:34.172-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 558 2016-04-07T03:48:34.202-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 558 2016-04-07T03:48:34.232-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 558 2016-04-07T03:48:34.245-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.78µs CommitTime 323.676763ms TotalFlushTime 8m11.065082639s TotalCommitTime 31.097547453s 2016-04-07T03:48:34.245-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 12524349, 12524349, 91 committed:true 2016-04-07T03:48:34.245-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 12524349, 12524349, 91 committed:true) SnapCreateDur 334.160522ms SnapOpenDur 35.493µs 2016-04-07T03:48:34.262-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 558 2016-04-07T03:48:34.292-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 558 2016-04-07T03:48:34.322-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 558 2016-04-07T03:48:34.352-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 558 2016-04-07T03:48:34.382-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 558 2016-04-07T03:48:34.412-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 558 2016-04-07T03:48:34.442-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 558 2016-04-07T03:48:34.472-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 558 2016-04-07T03:48:34.496-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:34.499-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12530000 QueuedCount 49794 2016-04-07T03:48:34.502-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 558 2016-04-07T03:48:34.532-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 558 2016-04-07T03:48:34.562-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 558 2016-04-07T03:48:34.592-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 558 2016-04-07T03:48:34.622-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 558 2016-04-07T03:48:34.652-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 558 2016-04-07T03:48:34.682-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 558 2016-04-07T03:48:34.712-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 558 2016-04-07T03:48:34.742-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 558 2016-04-07T03:48:34.772-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 558 2016-04-07T03:48:34.802-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 558 2016-04-07T03:48:34.832-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 558 2016-04-07T03:48:34.862-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 558 2016-04-07T03:48:34.892-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 558 2016-04-07T03:48:34.922-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 558 2016-04-07T03:48:34.952-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 558 2016-04-07T03:48:34.968-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12540000 QueuedCount 50000 2016-04-07T03:48:34.982-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 558 2016-04-07T03:48:35.012-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 558 2016-04-07T03:48:35.042-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 558 2016-04-07T03:48:35.072-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 558 2016-04-07T03:48:35.102-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 558 2016-04-07T03:48:35.132-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 558 2016-04-07T03:48:35.162-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 558 2016-04-07T03:48:35.192-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 558 2016-04-07T03:48:35.222-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 558 2016-04-07T03:48:35.252-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 558 2016-04-07T03:48:35.282-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 558 2016-04-07T03:48:35.312-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 558 2016-04-07T03:48:35.342-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 558 2016-04-07T03:48:35.372-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 558 2016-04-07T03:48:35.402-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 558 2016-04-07T03:48:35.432-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 558 2016-04-07T03:48:35.462-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 558 2016-04-07T03:48:35.463-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12550000 QueuedCount 50000 2016-04-07T03:48:35.492-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 558 2016-04-07T03:48:35.522-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 558 2016-04-07T03:48:35.552-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 558 2016-04-07T03:48:35.582-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 558 2016-04-07T03:48:35.612-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 558 2016-04-07T03:48:35.642-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 558 2016-04-07T03:48:35.672-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 558 2016-04-07T03:48:35.702-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 558 2016-04-07T03:48:35.732-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 558 2016-04-07T03:48:35.762-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 558 2016-04-07T03:48:35.792-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 558 2016-04-07T03:48:35.822-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 558 2016-04-07T03:48:35.852-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 558 2016-04-07T03:48:35.882-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8700 Milliseconds Vbucket 558 2016-04-07T03:48:35.909-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12560000 QueuedCount 50000 2016-04-07T03:48:35.912-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8730 Milliseconds Vbucket 558 2016-04-07T03:48:35.942-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8760 Milliseconds Vbucket 558 2016-04-07T03:48:35.972-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8790 Milliseconds Vbucket 558 2016-04-07T03:48:36.002-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8820 Milliseconds Vbucket 558 2016-04-07T03:48:36.032-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8850 Milliseconds Vbucket 558 2016-04-07T03:48:36.062-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8880 Milliseconds Vbucket 558 2016-04-07T03:48:36.092-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8910 Milliseconds Vbucket 558 2016-04-07T03:48:36.122-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8940 Milliseconds Vbucket 558 2016-04-07T03:48:36.152-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8970 Milliseconds Vbucket 558 2016-04-07T03:48:36.182-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9000 Milliseconds Vbucket 558 2016-04-07T03:48:36.212-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9030 Milliseconds Vbucket 558 2016-04-07T03:48:36.242-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9060 Milliseconds Vbucket 558 2016-04-07T03:48:36.272-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9090 Milliseconds Vbucket 558 2016-04-07T03:48:36.302-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9120 Milliseconds Vbucket 558 2016-04-07T03:48:36.312-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12570000 QueuedCount 50000 2016-04-07T03:48:36.332-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9150 Milliseconds Vbucket 558 2016-04-07T03:48:36.362-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9180 Milliseconds Vbucket 558 2016-04-07T03:48:36.392-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9210 Milliseconds Vbucket 558 2016-04-07T03:48:36.422-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9240 Milliseconds Vbucket 558 2016-04-07T03:48:36.452-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9270 Milliseconds Vbucket 558 2016-04-07T03:48:36.482-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9300 Milliseconds Vbucket 558 2016-04-07T03:48:36.512-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9330 Milliseconds Vbucket 558 2016-04-07T03:48:36.542-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9360 Milliseconds Vbucket 558 2016-04-07T03:48:36.572-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9390 Milliseconds Vbucket 558 2016-04-07T03:48:36.602-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9420 Milliseconds Vbucket 558 2016-04-07T03:48:36.632-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9450 Milliseconds Vbucket 558 2016-04-07T03:48:36.662-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9480 Milliseconds Vbucket 558 2016-04-07T03:48:36.692-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9510 Milliseconds Vbucket 558 2016-04-07T03:48:36.701-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12580000 QueuedCount 50000 2016-04-07T03:48:36.722-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9540 Milliseconds Vbucket 558 2016-04-07T03:48:36.752-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9570 Milliseconds Vbucket 558 2016-04-07T03:48:36.782-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9600 Milliseconds Vbucket 558 2016-04-07T03:48:36.812-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9630 Milliseconds Vbucket 558 2016-04-07T03:48:36.842-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9660 Milliseconds Vbucket 558 2016-04-07T03:48:36.866-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:36.872-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9690 Milliseconds Vbucket 558 2016-04-07T03:48:37.044-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19310000 2016-04-07T03:48:37.096-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12590000 QueuedCount 50000 2016-04-07T03:48:37.322-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19320000 2016-04-07T03:48:37.474-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12600000 QueuedCount 50000 2016-04-07T03:48:37.568-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19330000 2016-04-07T03:48:37.667-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19340000 2016-04-07T03:48:37.703-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19350000 2016-04-07T03:48:37.741-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19360000 2016-04-07T03:48:37.778-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19370000 2016-04-07T03:48:37.815-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19380000 2016-04-07T03:48:37.855-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19390000 2016-04-07T03:48:37.868-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12610000 QueuedCount 50000 2016-04-07T03:48:37.892-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19400000 2016-04-07T03:48:37.933-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19410000 2016-04-07T03:48:37.973-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19420000 2016-04-07T03:48:38.019-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19430000 2016-04-07T03:48:38.069-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19440000 2016-04-07T03:48:38.112-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19450000 2016-04-07T03:48:38.165-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19460000 2016-04-07T03:48:38.21-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19470000 2016-04-07T03:48:38.261-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19480000 2016-04-07T03:48:38.284-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12620000 QueuedCount 50000 2016-04-07T03:48:38.304-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19490000 2016-04-07T03:48:38.357-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19500000 2016-04-07T03:48:38.412-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19510000 2016-04-07T03:48:38.472-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19520000 2016-04-07T03:48:38.543-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19530000 2016-04-07T03:48:38.596-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19540000 2016-04-07T03:48:38.66-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19550000 2016-04-07T03:48:38.721-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12630000 QueuedCount 50000 2016-04-07T03:48:38.726-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19560000 2016-04-07T03:48:38.793-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19570000 2016-04-07T03:48:38.867-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19580000 2016-04-07T03:48:38.936-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19590000 2016-04-07T03:48:39-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19600000 2016-04-07T03:48:39.063-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19610000 2016-04-07T03:48:39.101-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12640000 QueuedCount 49162 2016-04-07T03:48:39.137-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19620000 2016-04-07T03:48:39.201-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19630000 2016-04-07T03:48:39.271-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19640000 2016-04-07T03:48:39.479-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12650000 QueuedCount 39162 2016-04-07T03:48:39.852-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12660000 QueuedCount 29162 2016-04-07T03:48:40.217-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12670000 QueuedCount 19162 2016-04-07T03:48:40.525-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12680000 QueuedCount 9162 2016-04-07T03:48:41.446-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.345µs CommitTime 552.688357ms TotalFlushTime 8m17.447324704s TotalCommitTime 31.65023581s 2016-04-07T03:48:41.446-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 12689162, 12689162, 92 committed:true 2016-04-07T03:48:41.446-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 12689162, 12689162, 92 committed:true) SnapCreateDur 564.397723ms SnapOpenDur 56.709µs 2016-04-07T03:48:41.487-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12690000 QueuedCount 34523 2016-04-07T03:48:41.793-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:41.982-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12700000 QueuedCount 50000 2016-04-07T03:48:42.447-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12710000 QueuedCount 50000 2016-04-07T03:48:42.921-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12720000 QueuedCount 50000 2016-04-07T03:48:43.327-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12730000 QueuedCount 50000 2016-04-07T03:48:44.206-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12740000 QueuedCount 50000 2016-04-07T03:48:44.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 595 2016-04-07T03:48:44.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 595 2016-04-07T03:48:44.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 595 2016-04-07T03:48:44.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 595 2016-04-07T03:48:44.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 595 2016-04-07T03:48:44.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 595 2016-04-07T03:48:44.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 595 2016-04-07T03:48:44.521-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 595 2016-04-07T03:48:44.551-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 595 2016-04-07T03:48:44.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 595 2016-04-07T03:48:44.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 595 2016-04-07T03:48:44.631-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:44.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 595 2016-04-07T03:48:44.644-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12750000 QueuedCount 49031 2016-04-07T03:48:44.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 595 2016-04-07T03:48:44.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 595 2016-04-07T03:48:44.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 595 2016-04-07T03:48:44.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 595 2016-04-07T03:48:44.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 595 2016-04-07T03:48:44.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 595 2016-04-07T03:48:44.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 595 2016-04-07T03:48:44.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 595 2016-04-07T03:48:44.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 595 2016-04-07T03:48:44.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 595 2016-04-07T03:48:44.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 595 2016-04-07T03:48:45.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 595 2016-04-07T03:48:45.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 595 2016-04-07T03:48:45.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 595 2016-04-07T03:48:45.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 595 2016-04-07T03:48:45.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 595 2016-04-07T03:48:45.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 595 2016-04-07T03:48:45.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 595 2016-04-07T03:48:45.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 595 2016-04-07T03:48:45.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 595 2016-04-07T03:48:45.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 595 2016-04-07T03:48:45.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 595 2016-04-07T03:48:45.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 595 2016-04-07T03:48:45.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 595 2016-04-07T03:48:45.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 595 2016-04-07T03:48:45.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 595 2016-04-07T03:48:45.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 595 2016-04-07T03:48:45.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 595 2016-04-07T03:48:45.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 595 2016-04-07T03:48:45.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 595 2016-04-07T03:48:45.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 595 2016-04-07T03:48:45.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 595 2016-04-07T03:48:45.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 595 2016-04-07T03:48:45.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 595 2016-04-07T03:48:45.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 595 2016-04-07T03:48:45.707-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12760000 QueuedCount 50000 2016-04-07T03:48:45.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 595 2016-04-07T03:48:45.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 595 2016-04-07T03:48:45.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 595 2016-04-07T03:48:45.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 595 2016-04-07T03:48:45.841-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 595 2016-04-07T03:48:45.871-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 595 2016-04-07T03:48:45.901-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 595 2016-04-07T03:48:45.931-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 595 2016-04-07T03:48:45.961-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 595 2016-04-07T03:48:45.991-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 595 2016-04-07T03:48:46.021-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 595 2016-04-07T03:48:46.051-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 595 2016-04-07T03:48:46.081-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 595 2016-04-07T03:48:46.111-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 595 2016-04-07T03:48:46.141-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 595 2016-04-07T03:48:46.171-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 595 2016-04-07T03:48:46.201-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 595 2016-04-07T03:48:46.231-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 595 2016-04-07T03:48:46.261-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 595 2016-04-07T03:48:46.291-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 595 2016-04-07T03:48:46.321-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 595 2016-04-07T03:48:46.351-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 595 2016-04-07T03:48:46.381-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 595 2016-04-07T03:48:46.411-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 595 2016-04-07T03:48:46.441-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 595 2016-04-07T03:48:46.471-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 595 2016-04-07T03:48:46.501-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 595 2016-04-07T03:48:46.531-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 595 2016-04-07T03:48:46.561-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 595 2016-04-07T03:48:46.591-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 595 2016-04-07T03:48:46.621-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 595 2016-04-07T03:48:46.651-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 595 2016-04-07T03:48:46.681-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 595 2016-04-07T03:48:46.711-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 595 2016-04-07T03:48:46.741-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 595 2016-04-07T03:48:46.771-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 595 2016-04-07T03:48:46.801-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 595 2016-04-07T03:48:46.831-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 595 2016-04-07T03:48:46.861-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 595 2016-04-07T03:48:46.891-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 595 2016-04-07T03:48:46.921-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 595 2016-04-07T03:48:46.951-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 595 2016-04-07T03:48:46.981-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 595 2016-04-07T03:48:47.011-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 595 2016-04-07T03:48:47.041-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 595 2016-04-07T03:48:47.071-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 595 2016-04-07T03:48:47.101-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 595 2016-04-07T03:48:47.131-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 595 2016-04-07T03:48:47.161-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 595 2016-04-07T03:48:47.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 595 2016-04-07T03:48:47.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 595 2016-04-07T03:48:47.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 595 2016-04-07T03:48:47.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 595 2016-04-07T03:48:47.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 595 2016-04-07T03:48:47.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 595 2016-04-07T03:48:47.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 595 2016-04-07T03:48:47.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 595 2016-04-07T03:48:47.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 595 2016-04-07T03:48:47.441-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12770000 QueuedCount 50000 2016-04-07T03:48:47.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 595 2016-04-07T03:48:47.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 595 2016-04-07T03:48:47.521-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 595 2016-04-07T03:48:47.551-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 595 2016-04-07T03:48:47.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 595 2016-04-07T03:48:47.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 595 2016-04-07T03:48:47.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 595 2016-04-07T03:48:47.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 595 2016-04-07T03:48:47.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 595 2016-04-07T03:48:47.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 595 2016-04-07T03:48:47.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 595 2016-04-07T03:48:47.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 595 2016-04-07T03:48:47.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 595 2016-04-07T03:48:47.84-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12780000 QueuedCount 50000 2016-04-07T03:48:47.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 595 2016-04-07T03:48:47.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 595 2016-04-07T03:48:47.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 595 2016-04-07T03:48:47.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 595 2016-04-07T03:48:47.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 595 2016-04-07T03:48:48.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8700 Milliseconds Vbucket 595 2016-04-07T03:48:48.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8730 Milliseconds Vbucket 595 2016-04-07T03:48:48.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8760 Milliseconds Vbucket 595 2016-04-07T03:48:48.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8790 Milliseconds Vbucket 595 2016-04-07T03:48:48.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8820 Milliseconds Vbucket 595 2016-04-07T03:48:48.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8850 Milliseconds Vbucket 595 2016-04-07T03:48:48.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8880 Milliseconds Vbucket 595 2016-04-07T03:48:48.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8910 Milliseconds Vbucket 595 2016-04-07T03:48:48.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8940 Milliseconds Vbucket 595 2016-04-07T03:48:48.244-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12790000 QueuedCount 50000 2016-04-07T03:48:48.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8970 Milliseconds Vbucket 595 2016-04-07T03:48:48.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9000 Milliseconds Vbucket 595 2016-04-07T03:48:48.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9030 Milliseconds Vbucket 595 2016-04-07T03:48:48.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9060 Milliseconds Vbucket 595 2016-04-07T03:48:48.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9090 Milliseconds Vbucket 595 2016-04-07T03:48:48.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9120 Milliseconds Vbucket 595 2016-04-07T03:48:48.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9150 Milliseconds Vbucket 595 2016-04-07T03:48:48.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9180 Milliseconds Vbucket 595 2016-04-07T03:48:48.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9210 Milliseconds Vbucket 595 2016-04-07T03:48:48.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9240 Milliseconds Vbucket 595 2016-04-07T03:48:48.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9270 Milliseconds Vbucket 595 2016-04-07T03:48:48.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9300 Milliseconds Vbucket 595 2016-04-07T03:48:48.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9330 Milliseconds Vbucket 595 2016-04-07T03:48:48.643-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12800000 QueuedCount 50000 2016-04-07T03:48:48.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9360 Milliseconds Vbucket 595 2016-04-07T03:48:48.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9390 Milliseconds Vbucket 595 2016-04-07T03:48:48.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9420 Milliseconds Vbucket 595 2016-04-07T03:48:48.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9450 Milliseconds Vbucket 595 2016-04-07T03:48:48.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9480 Milliseconds Vbucket 595 2016-04-07T03:48:48.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9510 Milliseconds Vbucket 595 2016-04-07T03:48:48.841-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9540 Milliseconds Vbucket 595 2016-04-07T03:48:48.871-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9570 Milliseconds Vbucket 595 2016-04-07T03:48:48.901-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9600 Milliseconds Vbucket 595 2016-04-07T03:48:48.931-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9630 Milliseconds Vbucket 595 2016-04-07T03:48:48.961-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9660 Milliseconds Vbucket 595 2016-04-07T03:48:48.972-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12810000 QueuedCount 45664 2016-04-07T03:48:48.991-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9690 Milliseconds Vbucket 595 2016-04-07T03:48:49.021-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9720 Milliseconds Vbucket 595 2016-04-07T03:48:49.051-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9750 Milliseconds Vbucket 595 2016-04-07T03:48:49.081-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9780 Milliseconds Vbucket 595 2016-04-07T03:48:49.111-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9810 Milliseconds Vbucket 595 2016-04-07T03:48:49.141-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9840 Milliseconds Vbucket 595 2016-04-07T03:48:49.171-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9870 Milliseconds Vbucket 595 2016-04-07T03:48:49.201-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9900 Milliseconds Vbucket 595 2016-04-07T03:48:49.231-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9930 Milliseconds Vbucket 595 2016-04-07T03:48:49.261-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9960 Milliseconds Vbucket 595 2016-04-07T03:48:49.291-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9990 Milliseconds Vbucket 595 2016-04-07T03:48:49.321-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10020 Milliseconds Vbucket 595 2016-04-07T03:48:49.328-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12820000 QueuedCount 35664 2016-04-07T03:48:49.351-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10050 Milliseconds Vbucket 595 2016-04-07T03:48:49.381-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10080 Milliseconds Vbucket 595 2016-04-07T03:48:49.411-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10110 Milliseconds Vbucket 595 2016-04-07T03:48:49.441-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10140 Milliseconds Vbucket 595 2016-04-07T03:48:49.471-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10170 Milliseconds Vbucket 595 2016-04-07T03:48:49.501-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10200 Milliseconds Vbucket 595 2016-04-07T03:48:49.531-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10230 Milliseconds Vbucket 595 2016-04-07T03:48:49.561-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10260 Milliseconds Vbucket 595 2016-04-07T03:48:49.591-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10290 Milliseconds Vbucket 595 2016-04-07T03:48:49.621-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10320 Milliseconds Vbucket 595 2016-04-07T03:48:49.651-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10350 Milliseconds Vbucket 595 2016-04-07T03:48:49.679-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12830000 QueuedCount 25664 2016-04-07T03:48:49.681-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10380 Milliseconds Vbucket 595 2016-04-07T03:48:49.711-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10410 Milliseconds Vbucket 595 2016-04-07T03:48:49.741-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10440 Milliseconds Vbucket 595 2016-04-07T03:48:49.771-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10470 Milliseconds Vbucket 595 2016-04-07T03:48:49.801-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10500 Milliseconds Vbucket 595 2016-04-07T03:48:49.831-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10530 Milliseconds Vbucket 595 2016-04-07T03:48:49.861-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10560 Milliseconds Vbucket 595 2016-04-07T03:48:49.891-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10590 Milliseconds Vbucket 595 2016-04-07T03:48:49.921-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10620 Milliseconds Vbucket 595 2016-04-07T03:48:49.951-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10650 Milliseconds Vbucket 595 2016-04-07T03:48:49.981-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10680 Milliseconds Vbucket 595 2016-04-07T03:48:50.011-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10710 Milliseconds Vbucket 595 2016-04-07T03:48:50.039-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12840000 QueuedCount 15664 2016-04-07T03:48:50.041-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10740 Milliseconds Vbucket 595 2016-04-07T03:48:50.071-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10770 Milliseconds Vbucket 595 2016-04-07T03:48:50.101-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10800 Milliseconds Vbucket 595 2016-04-07T03:48:50.131-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10830 Milliseconds Vbucket 595 2016-04-07T03:48:50.161-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10860 Milliseconds Vbucket 595 2016-04-07T03:48:50.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10890 Milliseconds Vbucket 595 2016-04-07T03:48:50.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10920 Milliseconds Vbucket 595 2016-04-07T03:48:50.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10950 Milliseconds Vbucket 595 2016-04-07T03:48:50.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10980 Milliseconds Vbucket 595 2016-04-07T03:48:50.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11010 Milliseconds Vbucket 595 2016-04-07T03:48:50.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11040 Milliseconds Vbucket 595 2016-04-07T03:48:50.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11070 Milliseconds Vbucket 595 2016-04-07T03:48:50.382-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12850000 QueuedCount 5664 2016-04-07T03:48:50.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11100 Milliseconds Vbucket 595 2016-04-07T03:48:50.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11130 Milliseconds Vbucket 595 2016-04-07T03:48:50.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11160 Milliseconds Vbucket 595 2016-04-07T03:48:50.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11190 Milliseconds Vbucket 595 2016-04-07T03:48:50.521-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11220 Milliseconds Vbucket 595 2016-04-07T03:48:50.551-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11250 Milliseconds Vbucket 595 2016-04-07T03:48:50.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11280 Milliseconds Vbucket 595 2016-04-07T03:48:50.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11310 Milliseconds Vbucket 595 2016-04-07T03:48:50.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11340 Milliseconds Vbucket 595 2016-04-07T03:48:50.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11370 Milliseconds Vbucket 595 2016-04-07T03:48:50.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11400 Milliseconds Vbucket 595 2016-04-07T03:48:50.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11430 Milliseconds Vbucket 595 2016-04-07T03:48:50.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11460 Milliseconds Vbucket 595 2016-04-07T03:48:50.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11490 Milliseconds Vbucket 595 2016-04-07T03:48:50.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11520 Milliseconds Vbucket 595 2016-04-07T03:48:50.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11550 Milliseconds Vbucket 595 2016-04-07T03:48:50.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11580 Milliseconds Vbucket 595 2016-04-07T03:48:50.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11610 Milliseconds Vbucket 595 2016-04-07T03:48:50.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11640 Milliseconds Vbucket 595 2016-04-07T03:48:50.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11670 Milliseconds Vbucket 595 2016-04-07T03:48:51.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11700 Milliseconds Vbucket 595 2016-04-07T03:48:51.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11730 Milliseconds Vbucket 595 2016-04-07T03:48:51.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11760 Milliseconds Vbucket 595 2016-04-07T03:48:51.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11790 Milliseconds Vbucket 595 2016-04-07T03:48:51.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11820 Milliseconds Vbucket 595 2016-04-07T03:48:51.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11850 Milliseconds Vbucket 595 2016-04-07T03:48:51.157-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.592µs CommitTime 531.676357ms TotalFlushTime 8m26.320034676s TotalCommitTime 32.181912167s 2016-04-07T03:48:51.157-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 12855664, 12855664, 93 committed:true 2016-04-07T03:48:51.157-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 12855664, 12855664, 93 committed:true) SnapCreateDur 542.131139ms SnapOpenDur 52.8µs 2016-04-07T03:48:51.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11880 Milliseconds Vbucket 595 2016-04-07T03:48:51.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11910 Milliseconds Vbucket 595 2016-04-07T03:48:51.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11940 Milliseconds Vbucket 595 2016-04-07T03:48:51.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11970 Milliseconds Vbucket 595 2016-04-07T03:48:51.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12000 Milliseconds Vbucket 595 2016-04-07T03:48:51.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12030 Milliseconds Vbucket 595 2016-04-07T03:48:51.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12060 Milliseconds Vbucket 595 2016-04-07T03:48:51.384-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12860000 QueuedCount 50000 2016-04-07T03:48:51.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12090 Milliseconds Vbucket 595 2016-04-07T03:48:51.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12120 Milliseconds Vbucket 595 2016-04-07T03:48:51.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12150 Milliseconds Vbucket 595 2016-04-07T03:48:51.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12180 Milliseconds Vbucket 595 2016-04-07T03:48:51.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12210 Milliseconds Vbucket 595 2016-04-07T03:48:51.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12240 Milliseconds Vbucket 595 2016-04-07T03:48:51.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12270 Milliseconds Vbucket 595 2016-04-07T03:48:51.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12300 Milliseconds Vbucket 595 2016-04-07T03:48:51.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12330 Milliseconds Vbucket 595 2016-04-07T03:48:51.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12360 Milliseconds Vbucket 595 2016-04-07T03:48:51.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12390 Milliseconds Vbucket 595 2016-04-07T03:48:51.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12420 Milliseconds Vbucket 595 2016-04-07T03:48:51.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12450 Milliseconds Vbucket 595 2016-04-07T03:48:51.769-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12870000 QueuedCount 40496 2016-04-07T03:48:51.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12480 Milliseconds Vbucket 595 2016-04-07T03:48:51.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12510 Milliseconds Vbucket 595 2016-04-07T03:48:51.841-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12540 Milliseconds Vbucket 595 2016-04-07T03:48:51.871-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12570 Milliseconds Vbucket 595 2016-04-07T03:48:51.901-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12600 Milliseconds Vbucket 595 2016-04-07T03:48:51.931-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12630 Milliseconds Vbucket 595 2016-04-07T03:48:51.961-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12660 Milliseconds Vbucket 595 2016-04-07T03:48:51.991-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12690 Milliseconds Vbucket 595 2016-04-07T03:48:52.021-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12720 Milliseconds Vbucket 595 2016-04-07T03:48:52.051-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12750 Milliseconds Vbucket 595 2016-04-07T03:48:52.081-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12780 Milliseconds Vbucket 595 2016-04-07T03:48:52.111-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12810 Milliseconds Vbucket 595 2016-04-07T03:48:52.141-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12840 Milliseconds Vbucket 595 2016-04-07T03:48:52.171-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12870 Milliseconds Vbucket 595 2016-04-07T03:48:52.194-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12880000 QueuedCount 30496 2016-04-07T03:48:52.201-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12900 Milliseconds Vbucket 595 2016-04-07T03:48:52.231-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12930 Milliseconds Vbucket 595 2016-04-07T03:48:52.261-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12960 Milliseconds Vbucket 595 2016-04-07T03:48:52.291-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12990 Milliseconds Vbucket 595 2016-04-07T03:48:52.321-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13020 Milliseconds Vbucket 595 2016-04-07T03:48:52.351-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13050 Milliseconds Vbucket 595 2016-04-07T03:48:52.381-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13080 Milliseconds Vbucket 595 2016-04-07T03:48:52.411-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13110 Milliseconds Vbucket 595 2016-04-07T03:48:52.441-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13140 Milliseconds Vbucket 595 2016-04-07T03:48:52.471-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13170 Milliseconds Vbucket 595 2016-04-07T03:48:52.501-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13200 Milliseconds Vbucket 595 2016-04-07T03:48:52.531-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13230 Milliseconds Vbucket 595 2016-04-07T03:48:52.561-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13260 Milliseconds Vbucket 595 2016-04-07T03:48:52.591-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13290 Milliseconds Vbucket 595 2016-04-07T03:48:52.598-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12890000 QueuedCount 20496 2016-04-07T03:48:52.621-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13320 Milliseconds Vbucket 595 2016-04-07T03:48:52.651-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13350 Milliseconds Vbucket 595 2016-04-07T03:48:52.681-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13380 Milliseconds Vbucket 595 2016-04-07T03:48:52.711-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13410 Milliseconds Vbucket 595 2016-04-07T03:48:52.741-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13440 Milliseconds Vbucket 595 2016-04-07T03:48:52.771-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13470 Milliseconds Vbucket 595 2016-04-07T03:48:52.801-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13500 Milliseconds Vbucket 595 2016-04-07T03:48:52.831-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13530 Milliseconds Vbucket 595 2016-04-07T03:48:52.861-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13560 Milliseconds Vbucket 595 2016-04-07T03:48:52.891-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13590 Milliseconds Vbucket 595 2016-04-07T03:48:52.921-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13620 Milliseconds Vbucket 595 2016-04-07T03:48:52.951-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13650 Milliseconds Vbucket 595 2016-04-07T03:48:52.965-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12900000 QueuedCount 10496 2016-04-07T03:48:52.981-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13680 Milliseconds Vbucket 595 2016-04-07T03:48:53.011-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13710 Milliseconds Vbucket 595 2016-04-07T03:48:53.041-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13740 Milliseconds Vbucket 595 2016-04-07T03:48:53.071-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13770 Milliseconds Vbucket 595 2016-04-07T03:48:53.101-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13800 Milliseconds Vbucket 595 2016-04-07T03:48:53.131-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13830 Milliseconds Vbucket 595 2016-04-07T03:48:53.161-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13860 Milliseconds Vbucket 595 2016-04-07T03:48:53.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13890 Milliseconds Vbucket 595 2016-04-07T03:48:53.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13920 Milliseconds Vbucket 595 2016-04-07T03:48:53.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13950 Milliseconds Vbucket 595 2016-04-07T03:48:53.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13980 Milliseconds Vbucket 595 2016-04-07T03:48:53.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14010 Milliseconds Vbucket 595 2016-04-07T03:48:53.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14040 Milliseconds Vbucket 595 2016-04-07T03:48:53.36-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12910000 QueuedCount 496 2016-04-07T03:48:53.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14070 Milliseconds Vbucket 595 2016-04-07T03:48:53.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14100 Milliseconds Vbucket 595 2016-04-07T03:48:53.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14130 Milliseconds Vbucket 595 2016-04-07T03:48:53.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14160 Milliseconds Vbucket 595 2016-04-07T03:48:53.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14190 Milliseconds Vbucket 595 2016-04-07T03:48:53.521-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14220 Milliseconds Vbucket 595 2016-04-07T03:48:53.551-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14250 Milliseconds Vbucket 595 2016-04-07T03:48:53.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14280 Milliseconds Vbucket 595 2016-04-07T03:48:53.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14310 Milliseconds Vbucket 595 2016-04-07T03:48:53.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14340 Milliseconds Vbucket 595 2016-04-07T03:48:53.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14370 Milliseconds Vbucket 595 2016-04-07T03:48:53.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14400 Milliseconds Vbucket 595 2016-04-07T03:48:53.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14430 Milliseconds Vbucket 595 2016-04-07T03:48:53.75-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.439µs CommitTime 369.622611ms TotalFlushTime 8m28.474669713s TotalCommitTime 32.551534778s 2016-04-07T03:48:53.75-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 12910496, 12910496, 94 committed:true 2016-04-07T03:48:53.75-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 12910496, 12910496, 94 committed:true) SnapCreateDur 379.44886ms SnapOpenDur 32.413µs 2016-04-07T03:48:53.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14460 Milliseconds Vbucket 595 2016-04-07T03:48:53.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14490 Milliseconds Vbucket 595 2016-04-07T03:48:53.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14520 Milliseconds Vbucket 595 2016-04-07T03:48:53.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14550 Milliseconds Vbucket 595 2016-04-07T03:48:53.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14580 Milliseconds Vbucket 595 2016-04-07T03:48:53.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14610 Milliseconds Vbucket 595 2016-04-07T03:48:53.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14640 Milliseconds Vbucket 595 2016-04-07T03:48:53.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14670 Milliseconds Vbucket 595 2016-04-07T03:48:53.986-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:54.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14700 Milliseconds Vbucket 595 2016-04-07T03:48:54.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14730 Milliseconds Vbucket 595 2016-04-07T03:48:54.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14760 Milliseconds Vbucket 595 2016-04-07T03:48:54.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14790 Milliseconds Vbucket 595 2016-04-07T03:48:54.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14820 Milliseconds Vbucket 595 2016-04-07T03:48:54.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14850 Milliseconds Vbucket 595 2016-04-07T03:48:54.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14880 Milliseconds Vbucket 595 2016-04-07T03:48:54.184-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12920000 QueuedCount 50000 2016-04-07T03:48:54.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14910 Milliseconds Vbucket 595 2016-04-07T03:48:54.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14940 Milliseconds Vbucket 595 2016-04-07T03:48:54.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14970 Milliseconds Vbucket 595 2016-04-07T03:48:54.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15000 Milliseconds Vbucket 595 2016-04-07T03:48:54.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15030 Milliseconds Vbucket 595 2016-04-07T03:48:54.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15060 Milliseconds Vbucket 595 2016-04-07T03:48:54.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15090 Milliseconds Vbucket 595 2016-04-07T03:48:54.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15120 Milliseconds Vbucket 595 2016-04-07T03:48:54.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15150 Milliseconds Vbucket 595 2016-04-07T03:48:54.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15180 Milliseconds Vbucket 595 2016-04-07T03:48:54.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15210 Milliseconds Vbucket 595 2016-04-07T03:48:54.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15240 Milliseconds Vbucket 595 2016-04-07T03:48:54.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15270 Milliseconds Vbucket 595 2016-04-07T03:48:54.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15300 Milliseconds Vbucket 595 2016-04-07T03:48:54.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15330 Milliseconds Vbucket 595 2016-04-07T03:48:54.661-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12930000 QueuedCount 50000 2016-04-07T03:48:54.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15360 Milliseconds Vbucket 595 2016-04-07T03:48:54.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15390 Milliseconds Vbucket 595 2016-04-07T03:48:54.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15420 Milliseconds Vbucket 595 2016-04-07T03:48:54.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15450 Milliseconds Vbucket 595 2016-04-07T03:48:54.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15480 Milliseconds Vbucket 595 2016-04-07T03:48:54.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15510 Milliseconds Vbucket 595 2016-04-07T03:48:54.841-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15540 Milliseconds Vbucket 595 2016-04-07T03:48:54.871-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15570 Milliseconds Vbucket 595 2016-04-07T03:48:54.901-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15600 Milliseconds Vbucket 595 2016-04-07T03:48:54.931-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15630 Milliseconds Vbucket 595 2016-04-07T03:48:54.961-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15660 Milliseconds Vbucket 595 2016-04-07T03:48:54.991-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15690 Milliseconds Vbucket 595 2016-04-07T03:48:55.021-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15720 Milliseconds Vbucket 595 2016-04-07T03:48:55.051-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15750 Milliseconds Vbucket 595 2016-04-07T03:48:55.081-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15780 Milliseconds Vbucket 595 2016-04-07T03:48:55.1-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12940000 QueuedCount 50000 2016-04-07T03:48:55.111-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15810 Milliseconds Vbucket 595 2016-04-07T03:48:55.141-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15840 Milliseconds Vbucket 595 2016-04-07T03:48:55.171-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15870 Milliseconds Vbucket 595 2016-04-07T03:48:55.201-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15900 Milliseconds Vbucket 595 2016-04-07T03:48:55.231-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15930 Milliseconds Vbucket 595 2016-04-07T03:48:55.261-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15960 Milliseconds Vbucket 595 2016-04-07T03:48:55.291-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15990 Milliseconds Vbucket 595 2016-04-07T03:48:55.321-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16020 Milliseconds Vbucket 595 2016-04-07T03:48:55.351-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16050 Milliseconds Vbucket 595 2016-04-07T03:48:55.381-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16080 Milliseconds Vbucket 595 2016-04-07T03:48:55.411-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16110 Milliseconds Vbucket 595 2016-04-07T03:48:55.441-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16140 Milliseconds Vbucket 595 2016-04-07T03:48:55.471-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16170 Milliseconds Vbucket 595 2016-04-07T03:48:55.501-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16200 Milliseconds Vbucket 595 2016-04-07T03:48:55.531-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16230 Milliseconds Vbucket 595 2016-04-07T03:48:55.541-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12950000 QueuedCount 50000 2016-04-07T03:48:55.561-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16260 Milliseconds Vbucket 595 2016-04-07T03:48:55.591-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16290 Milliseconds Vbucket 595 2016-04-07T03:48:55.621-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16320 Milliseconds Vbucket 595 2016-04-07T03:48:55.651-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16350 Milliseconds Vbucket 595 2016-04-07T03:48:55.681-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16380 Milliseconds Vbucket 595 2016-04-07T03:48:55.711-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16410 Milliseconds Vbucket 595 2016-04-07T03:48:55.741-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16440 Milliseconds Vbucket 595 2016-04-07T03:48:55.771-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16470 Milliseconds Vbucket 595 2016-04-07T03:48:55.801-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16500 Milliseconds Vbucket 595 2016-04-07T03:48:55.831-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16530 Milliseconds Vbucket 595 2016-04-07T03:48:55.861-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16560 Milliseconds Vbucket 595 2016-04-07T03:48:55.891-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16590 Milliseconds Vbucket 595 2016-04-07T03:48:55.921-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16620 Milliseconds Vbucket 595 2016-04-07T03:48:55.951-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16650 Milliseconds Vbucket 595 2016-04-07T03:48:55.981-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16680 Milliseconds Vbucket 595 2016-04-07T03:48:55.989-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12960000 QueuedCount 50000 2016-04-07T03:48:56.011-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16710 Milliseconds Vbucket 595 2016-04-07T03:48:56.041-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16740 Milliseconds Vbucket 595 2016-04-07T03:48:56.071-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16770 Milliseconds Vbucket 595 2016-04-07T03:48:56.101-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16800 Milliseconds Vbucket 595 2016-04-07T03:48:56.131-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16830 Milliseconds Vbucket 595 2016-04-07T03:48:56.161-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16860 Milliseconds Vbucket 595 2016-04-07T03:48:56.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16890 Milliseconds Vbucket 595 2016-04-07T03:48:56.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16920 Milliseconds Vbucket 595 2016-04-07T03:48:56.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16950 Milliseconds Vbucket 595 2016-04-07T03:48:56.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16980 Milliseconds Vbucket 595 2016-04-07T03:48:56.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17010 Milliseconds Vbucket 595 2016-04-07T03:48:56.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17040 Milliseconds Vbucket 595 2016-04-07T03:48:56.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17070 Milliseconds Vbucket 595 2016-04-07T03:48:56.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17100 Milliseconds Vbucket 595 2016-04-07T03:48:56.402-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12970000 QueuedCount 50000 2016-04-07T03:48:56.422-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:48:56.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17130 Milliseconds Vbucket 595 2016-04-07T03:48:56.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17160 Milliseconds Vbucket 595 2016-04-07T03:48:56.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17190 Milliseconds Vbucket 595 2016-04-07T03:48:56.521-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17220 Milliseconds Vbucket 595 2016-04-07T03:48:56.551-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17250 Milliseconds Vbucket 595 2016-04-07T03:48:56.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17280 Milliseconds Vbucket 595 2016-04-07T03:48:56.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17310 Milliseconds Vbucket 595 2016-04-07T03:48:56.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17340 Milliseconds Vbucket 595 2016-04-07T03:48:56.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17370 Milliseconds Vbucket 595 2016-04-07T03:48:56.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17400 Milliseconds Vbucket 595 2016-04-07T03:48:56.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17430 Milliseconds Vbucket 595 2016-04-07T03:48:56.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17460 Milliseconds Vbucket 595 2016-04-07T03:48:56.763-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12980000 QueuedCount 50000 2016-04-07T03:48:56.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17490 Milliseconds Vbucket 595 2016-04-07T03:48:56.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17520 Milliseconds Vbucket 595 2016-04-07T03:48:56.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17550 Milliseconds Vbucket 595 2016-04-07T03:48:56.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17580 Milliseconds Vbucket 595 2016-04-07T03:48:56.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17610 Milliseconds Vbucket 595 2016-04-07T03:48:56.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17640 Milliseconds Vbucket 595 2016-04-07T03:48:56.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17670 Milliseconds Vbucket 595 2016-04-07T03:48:57.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17700 Milliseconds Vbucket 595 2016-04-07T03:48:57.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17730 Milliseconds Vbucket 595 2016-04-07T03:48:57.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17760 Milliseconds Vbucket 595 2016-04-07T03:48:57.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17790 Milliseconds Vbucket 595 2016-04-07T03:48:57.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17820 Milliseconds Vbucket 595 2016-04-07T03:48:57.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17850 Milliseconds Vbucket 595 2016-04-07T03:48:57.171-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 12990000 QueuedCount 50000 2016-04-07T03:48:57.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17880 Milliseconds Vbucket 595 2016-04-07T03:48:57.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17910 Milliseconds Vbucket 595 2016-04-07T03:48:57.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17940 Milliseconds Vbucket 595 2016-04-07T03:48:57.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17970 Milliseconds Vbucket 595 2016-04-07T03:48:57.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18000 Milliseconds Vbucket 595 2016-04-07T03:48:57.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18030 Milliseconds Vbucket 595 2016-04-07T03:48:57.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18060 Milliseconds Vbucket 595 2016-04-07T03:48:57.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18090 Milliseconds Vbucket 595 2016-04-07T03:48:57.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18120 Milliseconds Vbucket 595 2016-04-07T03:48:57.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18150 Milliseconds Vbucket 595 2016-04-07T03:48:57.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18180 Milliseconds Vbucket 595 2016-04-07T03:48:57.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18210 Milliseconds Vbucket 595 2016-04-07T03:48:57.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18240 Milliseconds Vbucket 595 2016-04-07T03:48:57.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18270 Milliseconds Vbucket 595 2016-04-07T03:48:57.599-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13000000 QueuedCount 50000 2016-04-07T03:48:57.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18300 Milliseconds Vbucket 595 2016-04-07T03:48:57.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18330 Milliseconds Vbucket 595 2016-04-07T03:48:57.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18360 Milliseconds Vbucket 595 2016-04-07T03:48:57.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18390 Milliseconds Vbucket 595 2016-04-07T03:48:57.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18420 Milliseconds Vbucket 595 2016-04-07T03:48:57.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18450 Milliseconds Vbucket 595 2016-04-07T03:48:57.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18480 Milliseconds Vbucket 595 2016-04-07T03:48:57.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18510 Milliseconds Vbucket 595 2016-04-07T03:48:57.841-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18540 Milliseconds Vbucket 595 2016-04-07T03:48:57.871-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18570 Milliseconds Vbucket 595 2016-04-07T03:48:57.901-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18600 Milliseconds Vbucket 595 2016-04-07T03:48:57.931-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18630 Milliseconds Vbucket 595 2016-04-07T03:48:57.961-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18660 Milliseconds Vbucket 595 2016-04-07T03:48:57.991-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18690 Milliseconds Vbucket 595 2016-04-07T03:48:58.021-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18720 Milliseconds Vbucket 595 2016-04-07T03:48:58.036-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13010000 QueuedCount 50000 2016-04-07T03:48:58.051-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18750 Milliseconds Vbucket 595 2016-04-07T03:48:58.081-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18780 Milliseconds Vbucket 595 2016-04-07T03:48:58.111-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18810 Milliseconds Vbucket 595 2016-04-07T03:48:58.141-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18840 Milliseconds Vbucket 595 2016-04-07T03:48:58.171-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18870 Milliseconds Vbucket 595 2016-04-07T03:48:58.201-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18900 Milliseconds Vbucket 595 2016-04-07T03:48:58.231-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18930 Milliseconds Vbucket 595 2016-04-07T03:48:58.261-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18960 Milliseconds Vbucket 595 2016-04-07T03:48:58.291-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18990 Milliseconds Vbucket 595 2016-04-07T03:48:58.321-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19020 Milliseconds Vbucket 595 2016-04-07T03:48:58.351-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19050 Milliseconds Vbucket 595 2016-04-07T03:48:58.381-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19080 Milliseconds Vbucket 595 2016-04-07T03:48:58.411-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19110 Milliseconds Vbucket 595 2016-04-07T03:48:58.441-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19140 Milliseconds Vbucket 595 2016-04-07T03:48:58.453-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13020000 QueuedCount 50000 2016-04-07T03:48:58.471-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19170 Milliseconds Vbucket 595 2016-04-07T03:48:58.501-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19200 Milliseconds Vbucket 595 2016-04-07T03:48:58.531-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19230 Milliseconds Vbucket 595 2016-04-07T03:48:58.561-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19260 Milliseconds Vbucket 595 2016-04-07T03:48:58.591-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19290 Milliseconds Vbucket 595 2016-04-07T03:48:58.621-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19320 Milliseconds Vbucket 595 2016-04-07T03:48:58.651-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19350 Milliseconds Vbucket 595 2016-04-07T03:48:58.681-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19380 Milliseconds Vbucket 595 2016-04-07T03:48:58.711-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19410 Milliseconds Vbucket 595 2016-04-07T03:48:58.741-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19440 Milliseconds Vbucket 595 2016-04-07T03:48:58.771-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19470 Milliseconds Vbucket 595 2016-04-07T03:48:58.801-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19500 Milliseconds Vbucket 595 2016-04-07T03:48:58.831-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19530 Milliseconds Vbucket 595 2016-04-07T03:48:58.861-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19560 Milliseconds Vbucket 595 2016-04-07T03:48:58.865-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13030000 QueuedCount 45647 2016-04-07T03:48:58.891-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19590 Milliseconds Vbucket 595 2016-04-07T03:48:58.921-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19620 Milliseconds Vbucket 595 2016-04-07T03:48:58.951-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19650 Milliseconds Vbucket 595 2016-04-07T03:48:58.981-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19680 Milliseconds Vbucket 595 2016-04-07T03:48:59.011-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19710 Milliseconds Vbucket 595 2016-04-07T03:48:59.041-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19740 Milliseconds Vbucket 595 2016-04-07T03:48:59.071-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19770 Milliseconds Vbucket 595 2016-04-07T03:48:59.101-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19800 Milliseconds Vbucket 595 2016-04-07T03:48:59.131-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19830 Milliseconds Vbucket 595 2016-04-07T03:48:59.161-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19860 Milliseconds Vbucket 595 2016-04-07T03:48:59.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19890 Milliseconds Vbucket 595 2016-04-07T03:48:59.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19920 Milliseconds Vbucket 595 2016-04-07T03:48:59.238-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13040000 QueuedCount 35647 2016-04-07T03:48:59.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19950 Milliseconds Vbucket 595 2016-04-07T03:48:59.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19980 Milliseconds Vbucket 595 2016-04-07T03:48:59.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20010 Milliseconds Vbucket 595 2016-04-07T03:48:59.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20040 Milliseconds Vbucket 595 2016-04-07T03:48:59.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20070 Milliseconds Vbucket 595 2016-04-07T03:48:59.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20100 Milliseconds Vbucket 595 2016-04-07T03:48:59.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20130 Milliseconds Vbucket 595 2016-04-07T03:48:59.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20160 Milliseconds Vbucket 595 2016-04-07T03:48:59.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20190 Milliseconds Vbucket 595 2016-04-07T03:48:59.521-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20220 Milliseconds Vbucket 595 2016-04-07T03:48:59.551-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20250 Milliseconds Vbucket 595 2016-04-07T03:48:59.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20280 Milliseconds Vbucket 595 2016-04-07T03:48:59.605-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13050000 QueuedCount 25647 2016-04-07T03:48:59.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20310 Milliseconds Vbucket 595 2016-04-07T03:48:59.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20340 Milliseconds Vbucket 595 2016-04-07T03:48:59.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20370 Milliseconds Vbucket 595 2016-04-07T03:48:59.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20400 Milliseconds Vbucket 595 2016-04-07T03:48:59.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20430 Milliseconds Vbucket 595 2016-04-07T03:48:59.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20460 Milliseconds Vbucket 595 2016-04-07T03:48:59.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20490 Milliseconds Vbucket 595 2016-04-07T03:48:59.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20520 Milliseconds Vbucket 595 2016-04-07T03:48:59.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20550 Milliseconds Vbucket 595 2016-04-07T03:48:59.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20580 Milliseconds Vbucket 595 2016-04-07T03:48:59.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20610 Milliseconds Vbucket 595 2016-04-07T03:48:59.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20640 Milliseconds Vbucket 595 2016-04-07T03:48:59.962-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13060000 QueuedCount 15647 2016-04-07T03:48:59.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20670 Milliseconds Vbucket 595 2016-04-07T03:49:00.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20700 Milliseconds Vbucket 595 2016-04-07T03:49:00.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20730 Milliseconds Vbucket 595 2016-04-07T03:49:00.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20760 Milliseconds Vbucket 595 2016-04-07T03:49:00.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20790 Milliseconds Vbucket 595 2016-04-07T03:49:00.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20820 Milliseconds Vbucket 595 2016-04-07T03:49:00.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20850 Milliseconds Vbucket 595 2016-04-07T03:49:00.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20880 Milliseconds Vbucket 595 2016-04-07T03:49:00.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20910 Milliseconds Vbucket 595 2016-04-07T03:49:00.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20940 Milliseconds Vbucket 595 2016-04-07T03:49:00.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20970 Milliseconds Vbucket 595 2016-04-07T03:49:00.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21000 Milliseconds Vbucket 595 2016-04-07T03:49:00.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21030 Milliseconds Vbucket 595 2016-04-07T03:49:00.356-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13070000 QueuedCount 5647 2016-04-07T03:49:00.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21060 Milliseconds Vbucket 595 2016-04-07T03:49:00.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21090 Milliseconds Vbucket 595 2016-04-07T03:49:00.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21120 Milliseconds Vbucket 595 2016-04-07T03:49:00.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21150 Milliseconds Vbucket 595 2016-04-07T03:49:00.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21180 Milliseconds Vbucket 595 2016-04-07T03:49:00.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21210 Milliseconds Vbucket 595 2016-04-07T03:49:00.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21240 Milliseconds Vbucket 595 2016-04-07T03:49:00.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21270 Milliseconds Vbucket 595 2016-04-07T03:49:00.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21300 Milliseconds Vbucket 595 2016-04-07T03:49:00.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21330 Milliseconds Vbucket 595 2016-04-07T03:49:00.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21360 Milliseconds Vbucket 595 2016-04-07T03:49:00.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21390 Milliseconds Vbucket 595 2016-04-07T03:49:00.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21420 Milliseconds Vbucket 595 2016-04-07T03:49:00.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21450 Milliseconds Vbucket 595 2016-04-07T03:49:00.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21480 Milliseconds Vbucket 595 2016-04-07T03:49:00.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21510 Milliseconds Vbucket 595 2016-04-07T03:49:00.841-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21540 Milliseconds Vbucket 595 2016-04-07T03:49:00.871-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21570 Milliseconds Vbucket 595 2016-04-07T03:49:00.901-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21600 Milliseconds Vbucket 595 2016-04-07T03:49:00.931-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21630 Milliseconds Vbucket 595 2016-04-07T03:49:00.961-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21660 Milliseconds Vbucket 595 2016-04-07T03:49:00.991-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21690 Milliseconds Vbucket 595 2016-04-07T03:49:01.021-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21720 Milliseconds Vbucket 595 2016-04-07T03:49:01.051-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21750 Milliseconds Vbucket 595 2016-04-07T03:49:01.081-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21780 Milliseconds Vbucket 595 2016-04-07T03:49:01.111-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21810 Milliseconds Vbucket 595 2016-04-07T03:49:01.141-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21840 Milliseconds Vbucket 595 2016-04-07T03:49:01.168-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.331µs CommitTime 571.657038ms TotalFlushTime 8m35.016681348s TotalCommitTime 33.123191816s 2016-04-07T03:49:01.168-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 13075647, 13075647, 95 committed:true 2016-04-07T03:49:01.168-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 13075647, 13075647, 95 committed:true) SnapCreateDur 582.712709ms SnapOpenDur 45.814µs 2016-04-07T03:49:01.171-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21870 Milliseconds Vbucket 595 2016-04-07T03:49:01.201-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21900 Milliseconds Vbucket 595 2016-04-07T03:49:01.231-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21930 Milliseconds Vbucket 595 2016-04-07T03:49:01.261-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21960 Milliseconds Vbucket 595 2016-04-07T03:49:01.291-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21990 Milliseconds Vbucket 595 2016-04-07T03:49:01.321-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22020 Milliseconds Vbucket 595 2016-04-07T03:49:01.327-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:01.351-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22050 Milliseconds Vbucket 595 2016-04-07T03:49:01.381-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22080 Milliseconds Vbucket 595 2016-04-07T03:49:01.409-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13080000 QueuedCount 50000 2016-04-07T03:49:01.411-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22110 Milliseconds Vbucket 595 2016-04-07T03:49:01.441-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22140 Milliseconds Vbucket 595 2016-04-07T03:49:01.471-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22170 Milliseconds Vbucket 595 2016-04-07T03:49:01.501-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22200 Milliseconds Vbucket 595 2016-04-07T03:49:01.531-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22230 Milliseconds Vbucket 595 2016-04-07T03:49:01.561-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22260 Milliseconds Vbucket 595 2016-04-07T03:49:01.591-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22290 Milliseconds Vbucket 595 2016-04-07T03:49:01.621-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22320 Milliseconds Vbucket 595 2016-04-07T03:49:01.651-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22350 Milliseconds Vbucket 595 2016-04-07T03:49:01.681-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22380 Milliseconds Vbucket 595 2016-04-07T03:49:01.711-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22410 Milliseconds Vbucket 595 2016-04-07T03:49:01.741-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22440 Milliseconds Vbucket 595 2016-04-07T03:49:01.771-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22470 Milliseconds Vbucket 595 2016-04-07T03:49:01.801-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22500 Milliseconds Vbucket 595 2016-04-07T03:49:01.831-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22530 Milliseconds Vbucket 595 2016-04-07T03:49:01.861-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22560 Milliseconds Vbucket 595 2016-04-07T03:49:01.886-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13090000 QueuedCount 50000 2016-04-07T03:49:01.891-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22590 Milliseconds Vbucket 595 2016-04-07T03:49:01.921-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22620 Milliseconds Vbucket 595 2016-04-07T03:49:01.951-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22650 Milliseconds Vbucket 595 2016-04-07T03:49:01.981-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22680 Milliseconds Vbucket 595 2016-04-07T03:49:02.011-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22710 Milliseconds Vbucket 595 2016-04-07T03:49:02.041-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22740 Milliseconds Vbucket 595 2016-04-07T03:49:02.071-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22770 Milliseconds Vbucket 595 2016-04-07T03:49:02.101-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22800 Milliseconds Vbucket 595 2016-04-07T03:49:02.131-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22830 Milliseconds Vbucket 595 2016-04-07T03:49:02.161-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22860 Milliseconds Vbucket 595 2016-04-07T03:49:02.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22890 Milliseconds Vbucket 595 2016-04-07T03:49:02.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22920 Milliseconds Vbucket 595 2016-04-07T03:49:02.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22950 Milliseconds Vbucket 595 2016-04-07T03:49:02.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22980 Milliseconds Vbucket 595 2016-04-07T03:49:02.287-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13100000 QueuedCount 50000 2016-04-07T03:49:02.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23010 Milliseconds Vbucket 595 2016-04-07T03:49:02.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23040 Milliseconds Vbucket 595 2016-04-07T03:49:02.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23070 Milliseconds Vbucket 595 2016-04-07T03:49:02.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23100 Milliseconds Vbucket 595 2016-04-07T03:49:02.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23130 Milliseconds Vbucket 595 2016-04-07T03:49:02.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23160 Milliseconds Vbucket 595 2016-04-07T03:49:02.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23190 Milliseconds Vbucket 595 2016-04-07T03:49:02.521-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23220 Milliseconds Vbucket 595 2016-04-07T03:49:02.551-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23250 Milliseconds Vbucket 595 2016-04-07T03:49:02.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23280 Milliseconds Vbucket 595 2016-04-07T03:49:02.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23310 Milliseconds Vbucket 595 2016-04-07T03:49:02.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23340 Milliseconds Vbucket 595 2016-04-07T03:49:02.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23370 Milliseconds Vbucket 595 2016-04-07T03:49:02.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23400 Milliseconds Vbucket 595 2016-04-07T03:49:02.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23430 Milliseconds Vbucket 595 2016-04-07T03:49:02.747-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13110000 QueuedCount 50000 2016-04-07T03:49:02.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23460 Milliseconds Vbucket 595 2016-04-07T03:49:02.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23490 Milliseconds Vbucket 595 2016-04-07T03:49:02.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23520 Milliseconds Vbucket 595 2016-04-07T03:49:02.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23550 Milliseconds Vbucket 595 2016-04-07T03:49:02.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23580 Milliseconds Vbucket 595 2016-04-07T03:49:02.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23610 Milliseconds Vbucket 595 2016-04-07T03:49:02.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23640 Milliseconds Vbucket 595 2016-04-07T03:49:02.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23670 Milliseconds Vbucket 595 2016-04-07T03:49:03.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23700 Milliseconds Vbucket 595 2016-04-07T03:49:03.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23730 Milliseconds Vbucket 595 2016-04-07T03:49:03.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23760 Milliseconds Vbucket 595 2016-04-07T03:49:03.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23790 Milliseconds Vbucket 595 2016-04-07T03:49:03.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23820 Milliseconds Vbucket 595 2016-04-07T03:49:03.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23850 Milliseconds Vbucket 595 2016-04-07T03:49:03.158-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13120000 QueuedCount 50000 2016-04-07T03:49:03.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23880 Milliseconds Vbucket 595 2016-04-07T03:49:03.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23910 Milliseconds Vbucket 595 2016-04-07T03:49:03.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23940 Milliseconds Vbucket 595 2016-04-07T03:49:03.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23970 Milliseconds Vbucket 595 2016-04-07T03:49:03.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24000 Milliseconds Vbucket 595 2016-04-07T03:49:03.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24030 Milliseconds Vbucket 595 2016-04-07T03:49:03.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24060 Milliseconds Vbucket 595 2016-04-07T03:49:03.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24090 Milliseconds Vbucket 595 2016-04-07T03:49:03.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24120 Milliseconds Vbucket 595 2016-04-07T03:49:03.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24150 Milliseconds Vbucket 595 2016-04-07T03:49:03.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24180 Milliseconds Vbucket 595 2016-04-07T03:49:03.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24210 Milliseconds Vbucket 595 2016-04-07T03:49:03.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24240 Milliseconds Vbucket 595 2016-04-07T03:49:03.569-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13130000 QueuedCount 50000 2016-04-07T03:49:03.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24270 Milliseconds Vbucket 595 2016-04-07T03:49:03.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24300 Milliseconds Vbucket 595 2016-04-07T03:49:03.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24330 Milliseconds Vbucket 595 2016-04-07T03:49:03.651-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:03.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24360 Milliseconds Vbucket 595 2016-04-07T03:49:03.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24390 Milliseconds Vbucket 595 2016-04-07T03:49:03.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24420 Milliseconds Vbucket 595 2016-04-07T03:49:03.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24450 Milliseconds Vbucket 595 2016-04-07T03:49:03.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24480 Milliseconds Vbucket 595 2016-04-07T03:49:03.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24510 Milliseconds Vbucket 595 2016-04-07T03:49:03.841-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24540 Milliseconds Vbucket 595 2016-04-07T03:49:03.871-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24570 Milliseconds Vbucket 595 2016-04-07T03:49:03.901-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24600 Milliseconds Vbucket 595 2016-04-07T03:49:03.931-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24630 Milliseconds Vbucket 595 2016-04-07T03:49:03.961-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24660 Milliseconds Vbucket 595 2016-04-07T03:49:03.991-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24690 Milliseconds Vbucket 595 2016-04-07T03:49:04-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13140000 QueuedCount 50000 2016-04-07T03:49:04.021-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24720 Milliseconds Vbucket 595 2016-04-07T03:49:04.051-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24750 Milliseconds Vbucket 595 2016-04-07T03:49:04.081-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24780 Milliseconds Vbucket 595 2016-04-07T03:49:04.111-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24810 Milliseconds Vbucket 595 2016-04-07T03:49:04.141-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24840 Milliseconds Vbucket 595 2016-04-07T03:49:04.171-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24870 Milliseconds Vbucket 595 2016-04-07T03:49:04.201-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24900 Milliseconds Vbucket 595 2016-04-07T03:49:04.231-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24930 Milliseconds Vbucket 595 2016-04-07T03:49:04.261-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24960 Milliseconds Vbucket 595 2016-04-07T03:49:04.291-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24990 Milliseconds Vbucket 595 2016-04-07T03:49:04.321-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25020 Milliseconds Vbucket 595 2016-04-07T03:49:04.351-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25050 Milliseconds Vbucket 595 2016-04-07T03:49:04.381-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25080 Milliseconds Vbucket 595 2016-04-07T03:49:04.411-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25110 Milliseconds Vbucket 595 2016-04-07T03:49:04.423-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13150000 QueuedCount 50000 2016-04-07T03:49:04.441-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25140 Milliseconds Vbucket 595 2016-04-07T03:49:04.471-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25170 Milliseconds Vbucket 595 2016-04-07T03:49:04.501-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25200 Milliseconds Vbucket 595 2016-04-07T03:49:04.531-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25230 Milliseconds Vbucket 595 2016-04-07T03:49:04.561-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25260 Milliseconds Vbucket 595 2016-04-07T03:49:04.591-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25290 Milliseconds Vbucket 595 2016-04-07T03:49:04.621-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25320 Milliseconds Vbucket 595 2016-04-07T03:49:04.651-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25350 Milliseconds Vbucket 595 2016-04-07T03:49:04.681-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25380 Milliseconds Vbucket 595 2016-04-07T03:49:04.711-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25410 Milliseconds Vbucket 595 2016-04-07T03:49:04.741-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25440 Milliseconds Vbucket 595 2016-04-07T03:49:04.771-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25470 Milliseconds Vbucket 595 2016-04-07T03:49:04.801-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25500 Milliseconds Vbucket 595 2016-04-07T03:49:04.831-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25530 Milliseconds Vbucket 595 2016-04-07T03:49:04.861-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25560 Milliseconds Vbucket 595 2016-04-07T03:49:04.891-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25590 Milliseconds Vbucket 595 2016-04-07T03:49:04.913-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13160000 QueuedCount 50000 2016-04-07T03:49:04.921-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25620 Milliseconds Vbucket 595 2016-04-07T03:49:04.951-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25650 Milliseconds Vbucket 595 2016-04-07T03:49:04.981-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25680 Milliseconds Vbucket 595 2016-04-07T03:49:05.011-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25710 Milliseconds Vbucket 595 2016-04-07T03:49:05.041-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25740 Milliseconds Vbucket 595 2016-04-07T03:49:05.071-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25770 Milliseconds Vbucket 595 2016-04-07T03:49:05.101-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25800 Milliseconds Vbucket 595 2016-04-07T03:49:05.131-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25830 Milliseconds Vbucket 595 2016-04-07T03:49:05.161-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25860 Milliseconds Vbucket 595 2016-04-07T03:49:05.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25890 Milliseconds Vbucket 595 2016-04-07T03:49:05.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25920 Milliseconds Vbucket 595 2016-04-07T03:49:05.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25950 Milliseconds Vbucket 595 2016-04-07T03:49:05.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25980 Milliseconds Vbucket 595 2016-04-07T03:49:05.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26010 Milliseconds Vbucket 595 2016-04-07T03:49:05.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26040 Milliseconds Vbucket 595 2016-04-07T03:49:05.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26070 Milliseconds Vbucket 595 2016-04-07T03:49:05.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26100 Milliseconds Vbucket 595 2016-04-07T03:49:05.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26130 Milliseconds Vbucket 595 2016-04-07T03:49:05.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26160 Milliseconds Vbucket 595 2016-04-07T03:49:05.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26190 Milliseconds Vbucket 595 2016-04-07T03:49:05.521-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26220 Milliseconds Vbucket 595 2016-04-07T03:49:05.551-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26250 Milliseconds Vbucket 595 2016-04-07T03:49:05.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26280 Milliseconds Vbucket 595 2016-04-07T03:49:05.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26310 Milliseconds Vbucket 595 2016-04-07T03:49:05.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26340 Milliseconds Vbucket 595 2016-04-07T03:49:05.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26370 Milliseconds Vbucket 595 2016-04-07T03:49:05.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26400 Milliseconds Vbucket 595 2016-04-07T03:49:05.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26430 Milliseconds Vbucket 595 2016-04-07T03:49:05.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26460 Milliseconds Vbucket 595 2016-04-07T03:49:05.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26490 Milliseconds Vbucket 595 2016-04-07T03:49:05.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26520 Milliseconds Vbucket 595 2016-04-07T03:49:05.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26550 Milliseconds Vbucket 595 2016-04-07T03:49:05.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26580 Milliseconds Vbucket 595 2016-04-07T03:49:05.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26610 Milliseconds Vbucket 595 2016-04-07T03:49:05.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26640 Milliseconds Vbucket 595 2016-04-07T03:49:05.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26670 Milliseconds Vbucket 595 2016-04-07T03:49:06.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26700 Milliseconds Vbucket 595 2016-04-07T03:49:06.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26730 Milliseconds Vbucket 595 2016-04-07T03:49:06.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26760 Milliseconds Vbucket 595 2016-04-07T03:49:06.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26790 Milliseconds Vbucket 595 2016-04-07T03:49:06.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26820 Milliseconds Vbucket 595 2016-04-07T03:49:06.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26850 Milliseconds Vbucket 595 2016-04-07T03:49:06.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26880 Milliseconds Vbucket 595 2016-04-07T03:49:06.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26910 Milliseconds Vbucket 595 2016-04-07T03:49:06.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26940 Milliseconds Vbucket 595 2016-04-07T03:49:06.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26970 Milliseconds Vbucket 595 2016-04-07T03:49:06.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27000 Milliseconds Vbucket 595 2016-04-07T03:49:06.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27030 Milliseconds Vbucket 595 2016-04-07T03:49:06.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27060 Milliseconds Vbucket 595 2016-04-07T03:49:06.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27090 Milliseconds Vbucket 595 2016-04-07T03:49:06.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27120 Milliseconds Vbucket 595 2016-04-07T03:49:06.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27150 Milliseconds Vbucket 595 2016-04-07T03:49:06.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27180 Milliseconds Vbucket 595 2016-04-07T03:49:06.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27210 Milliseconds Vbucket 595 2016-04-07T03:49:06.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27240 Milliseconds Vbucket 595 2016-04-07T03:49:06.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27270 Milliseconds Vbucket 595 2016-04-07T03:49:06.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27300 Milliseconds Vbucket 595 2016-04-07T03:49:06.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27330 Milliseconds Vbucket 595 2016-04-07T03:49:06.644-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13170000 QueuedCount 50000 2016-04-07T03:49:06.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27360 Milliseconds Vbucket 595 2016-04-07T03:49:06.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27390 Milliseconds Vbucket 595 2016-04-07T03:49:06.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27420 Milliseconds Vbucket 595 2016-04-07T03:49:06.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27450 Milliseconds Vbucket 595 2016-04-07T03:49:06.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27480 Milliseconds Vbucket 595 2016-04-07T03:49:06.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27510 Milliseconds Vbucket 595 2016-04-07T03:49:06.841-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27540 Milliseconds Vbucket 595 2016-04-07T03:49:06.871-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27570 Milliseconds Vbucket 595 2016-04-07T03:49:06.901-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27600 Milliseconds Vbucket 595 2016-04-07T03:49:06.931-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27630 Milliseconds Vbucket 595 2016-04-07T03:49:06.961-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27660 Milliseconds Vbucket 595 2016-04-07T03:49:06.991-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27690 Milliseconds Vbucket 595 2016-04-07T03:49:07.021-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27720 Milliseconds Vbucket 595 2016-04-07T03:49:07.051-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27750 Milliseconds Vbucket 595 2016-04-07T03:49:07.081-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27780 Milliseconds Vbucket 595 2016-04-07T03:49:07.111-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27810 Milliseconds Vbucket 595 2016-04-07T03:49:07.141-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27840 Milliseconds Vbucket 595 2016-04-07T03:49:07.171-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27870 Milliseconds Vbucket 595 2016-04-07T03:49:07.201-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27900 Milliseconds Vbucket 595 2016-04-07T03:49:07.231-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27930 Milliseconds Vbucket 595 2016-04-07T03:49:07.261-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27960 Milliseconds Vbucket 595 2016-04-07T03:49:07.291-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27990 Milliseconds Vbucket 595 2016-04-07T03:49:07.321-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28020 Milliseconds Vbucket 595 2016-04-07T03:49:07.351-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28050 Milliseconds Vbucket 595 2016-04-07T03:49:07.381-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28080 Milliseconds Vbucket 595 2016-04-07T03:49:07.411-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28110 Milliseconds Vbucket 595 2016-04-07T03:49:07.441-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28140 Milliseconds Vbucket 595 2016-04-07T03:49:07.471-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28170 Milliseconds Vbucket 595 2016-04-07T03:49:07.501-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28200 Milliseconds Vbucket 595 2016-04-07T03:49:07.531-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28230 Milliseconds Vbucket 595 2016-04-07T03:49:07.553-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13180000 QueuedCount 50000 2016-04-07T03:49:07.561-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28260 Milliseconds Vbucket 595 2016-04-07T03:49:07.591-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28290 Milliseconds Vbucket 595 2016-04-07T03:49:07.621-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28320 Milliseconds Vbucket 595 2016-04-07T03:49:07.651-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28350 Milliseconds Vbucket 595 2016-04-07T03:49:07.681-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28380 Milliseconds Vbucket 595 2016-04-07T03:49:07.711-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28410 Milliseconds Vbucket 595 2016-04-07T03:49:07.741-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28440 Milliseconds Vbucket 595 2016-04-07T03:49:07.771-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28470 Milliseconds Vbucket 595 2016-04-07T03:49:07.801-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28500 Milliseconds Vbucket 595 2016-04-07T03:49:07.831-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28530 Milliseconds Vbucket 595 2016-04-07T03:49:07.861-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28560 Milliseconds Vbucket 595 2016-04-07T03:49:07.891-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28590 Milliseconds Vbucket 595 2016-04-07T03:49:07.921-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28620 Milliseconds Vbucket 595 2016-04-07T03:49:07.929-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13190000 QueuedCount 43718 2016-04-07T03:49:07.951-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28650 Milliseconds Vbucket 595 2016-04-07T03:49:07.981-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28680 Milliseconds Vbucket 595 2016-04-07T03:49:08.011-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28710 Milliseconds Vbucket 595 2016-04-07T03:49:08.041-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28740 Milliseconds Vbucket 595 2016-04-07T03:49:08.071-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28770 Milliseconds Vbucket 595 2016-04-07T03:49:08.101-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28800 Milliseconds Vbucket 595 2016-04-07T03:49:08.131-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28830 Milliseconds Vbucket 595 2016-04-07T03:49:08.161-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28860 Milliseconds Vbucket 595 2016-04-07T03:49:08.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28890 Milliseconds Vbucket 595 2016-04-07T03:49:08.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28920 Milliseconds Vbucket 595 2016-04-07T03:49:08.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28950 Milliseconds Vbucket 595 2016-04-07T03:49:08.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28980 Milliseconds Vbucket 595 2016-04-07T03:49:08.305-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13200000 QueuedCount 33718 2016-04-07T03:49:08.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29010 Milliseconds Vbucket 595 2016-04-07T03:49:08.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29040 Milliseconds Vbucket 595 2016-04-07T03:49:08.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29070 Milliseconds Vbucket 595 2016-04-07T03:49:08.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29100 Milliseconds Vbucket 595 2016-04-07T03:49:08.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29130 Milliseconds Vbucket 595 2016-04-07T03:49:08.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29160 Milliseconds Vbucket 595 2016-04-07T03:49:08.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29190 Milliseconds Vbucket 595 2016-04-07T03:49:08.521-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29220 Milliseconds Vbucket 595 2016-04-07T03:49:08.551-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29250 Milliseconds Vbucket 595 2016-04-07T03:49:08.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29280 Milliseconds Vbucket 595 2016-04-07T03:49:08.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29310 Milliseconds Vbucket 595 2016-04-07T03:49:08.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29340 Milliseconds Vbucket 595 2016-04-07T03:49:08.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29370 Milliseconds Vbucket 595 2016-04-07T03:49:08.686-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13210000 QueuedCount 23718 2016-04-07T03:49:08.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29400 Milliseconds Vbucket 595 2016-04-07T03:49:08.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29430 Milliseconds Vbucket 595 2016-04-07T03:49:08.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29460 Milliseconds Vbucket 595 2016-04-07T03:49:08.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29490 Milliseconds Vbucket 595 2016-04-07T03:49:08.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29520 Milliseconds Vbucket 595 2016-04-07T03:49:08.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29550 Milliseconds Vbucket 595 2016-04-07T03:49:08.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29580 Milliseconds Vbucket 595 2016-04-07T03:49:08.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29610 Milliseconds Vbucket 595 2016-04-07T03:49:08.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29640 Milliseconds Vbucket 595 2016-04-07T03:49:08.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29670 Milliseconds Vbucket 595 2016-04-07T03:49:09.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29700 Milliseconds Vbucket 595 2016-04-07T03:49:09.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29730 Milliseconds Vbucket 595 2016-04-07T03:49:09.054-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13220000 QueuedCount 13718 2016-04-07T03:49:09.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29760 Milliseconds Vbucket 595 2016-04-07T03:49:09.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29790 Milliseconds Vbucket 595 2016-04-07T03:49:09.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29820 Milliseconds Vbucket 595 2016-04-07T03:49:09.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29850 Milliseconds Vbucket 595 2016-04-07T03:49:09.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29880 Milliseconds Vbucket 595 2016-04-07T03:49:09.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29910 Milliseconds Vbucket 595 2016-04-07T03:49:09.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29940 Milliseconds Vbucket 595 2016-04-07T03:49:09.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29970 Milliseconds Vbucket 595 2016-04-07T03:49:09.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30000 Milliseconds Vbucket 595 2016-04-07T03:49:09.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30030 Milliseconds Vbucket 595 2016-04-07T03:49:09.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30060 Milliseconds Vbucket 595 2016-04-07T03:49:09.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30090 Milliseconds Vbucket 595 2016-04-07T03:49:09.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30120 Milliseconds Vbucket 595 2016-04-07T03:49:09.448-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13230000 QueuedCount 3718 2016-04-07T03:49:09.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30150 Milliseconds Vbucket 595 2016-04-07T03:49:09.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30180 Milliseconds Vbucket 595 2016-04-07T03:49:09.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30210 Milliseconds Vbucket 595 2016-04-07T03:49:09.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30240 Milliseconds Vbucket 595 2016-04-07T03:49:09.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30270 Milliseconds Vbucket 595 2016-04-07T03:49:09.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30300 Milliseconds Vbucket 595 2016-04-07T03:49:09.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30330 Milliseconds Vbucket 595 2016-04-07T03:49:09.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30360 Milliseconds Vbucket 595 2016-04-07T03:49:09.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30390 Milliseconds Vbucket 595 2016-04-07T03:49:09.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30420 Milliseconds Vbucket 595 2016-04-07T03:49:09.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30450 Milliseconds Vbucket 595 2016-04-07T03:49:09.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30480 Milliseconds Vbucket 595 2016-04-07T03:49:09.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30510 Milliseconds Vbucket 595 2016-04-07T03:49:09.841-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30540 Milliseconds Vbucket 595 2016-04-07T03:49:09.871-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30570 Milliseconds Vbucket 595 2016-04-07T03:49:09.901-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30600 Milliseconds Vbucket 595 2016-04-07T03:49:09.931-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30630 Milliseconds Vbucket 595 2016-04-07T03:49:09.961-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30660 Milliseconds Vbucket 595 2016-04-07T03:49:09.991-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30690 Milliseconds Vbucket 595 2016-04-07T03:49:10.021-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30720 Milliseconds Vbucket 595 2016-04-07T03:49:10.051-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30750 Milliseconds Vbucket 595 2016-04-07T03:49:10.081-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30780 Milliseconds Vbucket 595 2016-04-07T03:49:10.111-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30810 Milliseconds Vbucket 595 2016-04-07T03:49:10.135-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.214µs CommitTime 521.142376ms TotalFlushTime 8m43.1861126s TotalCommitTime 33.644334192s 2016-04-07T03:49:10.136-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 13233718, 13233718, 96 committed:true 2016-04-07T03:49:10.136-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 13233718, 13233718, 96 committed:true) SnapCreateDur 531.092668ms SnapOpenDur 43.37µs 2016-04-07T03:49:10.141-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30840 Milliseconds Vbucket 595 2016-04-07T03:49:10.171-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30870 Milliseconds Vbucket 595 2016-04-07T03:49:10.201-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30900 Milliseconds Vbucket 595 2016-04-07T03:49:10.231-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30930 Milliseconds Vbucket 595 2016-04-07T03:49:10.261-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30960 Milliseconds Vbucket 595 2016-04-07T03:49:10.291-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30990 Milliseconds Vbucket 595 2016-04-07T03:49:10.321-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31020 Milliseconds Vbucket 595 2016-04-07T03:49:10.351-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31050 Milliseconds Vbucket 595 2016-04-07T03:49:10.381-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31080 Milliseconds Vbucket 595 2016-04-07T03:49:10.411-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31110 Milliseconds Vbucket 595 2016-04-07T03:49:10.441-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31140 Milliseconds Vbucket 595 2016-04-07T03:49:10.471-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31170 Milliseconds Vbucket 595 2016-04-07T03:49:10.485-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13240000 QueuedCount 46598 2016-04-07T03:49:10.501-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31200 Milliseconds Vbucket 595 2016-04-07T03:49:10.531-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31230 Milliseconds Vbucket 595 2016-04-07T03:49:10.561-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31260 Milliseconds Vbucket 595 2016-04-07T03:49:10.591-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31290 Milliseconds Vbucket 595 2016-04-07T03:49:10.621-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31320 Milliseconds Vbucket 595 2016-04-07T03:49:10.651-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31350 Milliseconds Vbucket 595 2016-04-07T03:49:10.681-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31380 Milliseconds Vbucket 595 2016-04-07T03:49:10.711-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31410 Milliseconds Vbucket 595 2016-04-07T03:49:10.741-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31440 Milliseconds Vbucket 595 2016-04-07T03:49:10.771-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31470 Milliseconds Vbucket 595 2016-04-07T03:49:10.801-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31500 Milliseconds Vbucket 595 2016-04-07T03:49:10.831-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31530 Milliseconds Vbucket 595 2016-04-07T03:49:10.861-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31560 Milliseconds Vbucket 595 2016-04-07T03:49:10.878-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13250000 QueuedCount 36598 2016-04-07T03:49:10.891-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31590 Milliseconds Vbucket 595 2016-04-07T03:49:10.921-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31620 Milliseconds Vbucket 595 2016-04-07T03:49:10.951-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31650 Milliseconds Vbucket 595 2016-04-07T03:49:10.981-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31680 Milliseconds Vbucket 595 2016-04-07T03:49:11.011-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31710 Milliseconds Vbucket 595 2016-04-07T03:49:11.041-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31740 Milliseconds Vbucket 595 2016-04-07T03:49:11.071-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31770 Milliseconds Vbucket 595 2016-04-07T03:49:11.102-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31800 Milliseconds Vbucket 595 2016-04-07T03:49:11.131-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31830 Milliseconds Vbucket 595 2016-04-07T03:49:11.161-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31860 Milliseconds Vbucket 595 2016-04-07T03:49:11.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31890 Milliseconds Vbucket 595 2016-04-07T03:49:11.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31920 Milliseconds Vbucket 595 2016-04-07T03:49:11.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31950 Milliseconds Vbucket 595 2016-04-07T03:49:11.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31980 Milliseconds Vbucket 595 2016-04-07T03:49:11.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32010 Milliseconds Vbucket 595 2016-04-07T03:49:11.323-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13260000 QueuedCount 26598 2016-04-07T03:49:11.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32040 Milliseconds Vbucket 595 2016-04-07T03:49:11.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32070 Milliseconds Vbucket 595 2016-04-07T03:49:11.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32100 Milliseconds Vbucket 595 2016-04-07T03:49:11.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32130 Milliseconds Vbucket 595 2016-04-07T03:49:11.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32160 Milliseconds Vbucket 595 2016-04-07T03:49:11.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32190 Milliseconds Vbucket 595 2016-04-07T03:49:11.521-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32220 Milliseconds Vbucket 595 2016-04-07T03:49:11.551-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32250 Milliseconds Vbucket 595 2016-04-07T03:49:11.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32280 Milliseconds Vbucket 595 2016-04-07T03:49:11.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32310 Milliseconds Vbucket 595 2016-04-07T03:49:11.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32340 Milliseconds Vbucket 595 2016-04-07T03:49:11.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32370 Milliseconds Vbucket 595 2016-04-07T03:49:11.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32400 Milliseconds Vbucket 595 2016-04-07T03:49:11.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32430 Milliseconds Vbucket 595 2016-04-07T03:49:11.736-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13270000 QueuedCount 16598 2016-04-07T03:49:11.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32460 Milliseconds Vbucket 595 2016-04-07T03:49:11.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32490 Milliseconds Vbucket 595 2016-04-07T03:49:11.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32520 Milliseconds Vbucket 595 2016-04-07T03:49:11.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32550 Milliseconds Vbucket 595 2016-04-07T03:49:11.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32580 Milliseconds Vbucket 595 2016-04-07T03:49:11.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32610 Milliseconds Vbucket 595 2016-04-07T03:49:11.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32640 Milliseconds Vbucket 595 2016-04-07T03:49:11.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32670 Milliseconds Vbucket 595 2016-04-07T03:49:12.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32700 Milliseconds Vbucket 595 2016-04-07T03:49:12.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32730 Milliseconds Vbucket 595 2016-04-07T03:49:12.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32760 Milliseconds Vbucket 595 2016-04-07T03:49:12.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32790 Milliseconds Vbucket 595 2016-04-07T03:49:12.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32820 Milliseconds Vbucket 595 2016-04-07T03:49:12.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32850 Milliseconds Vbucket 595 2016-04-07T03:49:12.168-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13280000 QueuedCount 6598 2016-04-07T03:49:12.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32880 Milliseconds Vbucket 595 2016-04-07T03:49:12.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32910 Milliseconds Vbucket 595 2016-04-07T03:49:12.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32940 Milliseconds Vbucket 595 2016-04-07T03:49:12.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32970 Milliseconds Vbucket 595 2016-04-07T03:49:12.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33000 Milliseconds Vbucket 595 2016-04-07T03:49:12.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33030 Milliseconds Vbucket 595 2016-04-07T03:49:12.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33060 Milliseconds Vbucket 595 2016-04-07T03:49:12.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33090 Milliseconds Vbucket 595 2016-04-07T03:49:12.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33120 Milliseconds Vbucket 595 2016-04-07T03:49:12.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33150 Milliseconds Vbucket 595 2016-04-07T03:49:12.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33180 Milliseconds Vbucket 595 2016-04-07T03:49:12.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33210 Milliseconds Vbucket 595 2016-04-07T03:49:12.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33240 Milliseconds Vbucket 595 2016-04-07T03:49:12.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33270 Milliseconds Vbucket 595 2016-04-07T03:49:12.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33300 Milliseconds Vbucket 595 2016-04-07T03:49:12.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33330 Milliseconds Vbucket 595 2016-04-07T03:49:12.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33360 Milliseconds Vbucket 595 2016-04-07T03:49:12.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33390 Milliseconds Vbucket 595 2016-04-07T03:49:12.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33420 Milliseconds Vbucket 595 2016-04-07T03:49:12.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33450 Milliseconds Vbucket 595 2016-04-07T03:49:12.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33480 Milliseconds Vbucket 595 2016-04-07T03:49:12.802-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.036µs CommitTime 367.041549ms TotalFlushTime 8m45.42164662s TotalCommitTime 34.011375741s 2016-04-07T03:49:12.802-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 13286598, 13286598, 97 committed:true 2016-04-07T03:49:12.802-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 13286598, 13286598, 97 committed:true) SnapCreateDur 377.867286ms SnapOpenDur 50.065µs 2016-04-07T03:49:12.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33510 Milliseconds Vbucket 595 2016-04-07T03:49:12.86-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19650000 2016-04-07T03:49:12.904-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19660000 2016-04-07T03:49:12.943-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19670000 2016-04-07T03:49:12.943-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":6331215,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":5145289225,"bucket-1:myindex:avg_ts_items_count":99107,"bucket-1:myindex:build_progress":66,"bucket-1:myindex:data_size":4280105904,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":23342792704,"bucket-1:myindex:flush_queue_size":50000,"bucket-1:myindex:frag_percent":81,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":1390199895,"bucket-1:myindex:items_count":13233718,"bucket-1:myindex:num_commits":97,"bucket-1:myindex:num_compactions":2,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":13233718,"bucket-1:myindex:num_docs_pending":355754,"bucket-1:myindex:num_docs_queued":6410528,"bucket-1:myindex:num_flush_queued":13338887,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":97,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":2666182276,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"97 34011375741 -4909379429910375857","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"13288889 84680147466 610771793778555520","bucket-1:myindex:timings/storage_handle_open":"97 2317598 57420288472","bucket-1:myindex:timings/storage_info":"389 3668097 88764794855","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"98 68991229 729545118875013","bucket-1:myindex:timings/storage_meta_set":"97 365957764 107830372476853150","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"26577778 432233078763 -1804680784784798715","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":19670105,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":528,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"2341 282762311 47601147753893"} 2016-04-07T03:49:12.946-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:12.959-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13290000 QueuedCount 49083 2016-04-07T03:49:12.981-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19680000 2016-04-07T03:49:13.021-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19690000 2016-04-07T03:49:13.054-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19700000 2016-04-07T03:49:13.091-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19710000 2016-04-07T03:49:13.129-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19720000 2016-04-07T03:49:13.162-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19730000 2016-04-07T03:49:13.212-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19740000 2016-04-07T03:49:13.25-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19750000 2016-04-07T03:49:13.298-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19760000 2016-04-07T03:49:13.338-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19770000 2016-04-07T03:49:13.386-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19780000 2016-04-07T03:49:13.418-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13300000 QueuedCount 50000 2016-04-07T03:49:13.427-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19790000 2016-04-07T03:49:13.474-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19800000 2016-04-07T03:49:13.514-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19810000 2016-04-07T03:49:13.561-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19820000 2016-04-07T03:49:13.602-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19830000 2016-04-07T03:49:13.65-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19840000 2016-04-07T03:49:13.69-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19850000 2016-04-07T03:49:13.734-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19860000 2016-04-07T03:49:13.775-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19870000 2016-04-07T03:49:13.82-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19880000 2016-04-07T03:49:13.862-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13310000 QueuedCount 50000 2016-04-07T03:49:13.865-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19890000 2016-04-07T03:49:13.902-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19900000 2016-04-07T03:49:13.944-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19910000 2016-04-07T03:49:13.984-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19920000 2016-04-07T03:49:14.036-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19930000 2016-04-07T03:49:14.078-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19940000 2016-04-07T03:49:14.129-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19950000 2016-04-07T03:49:14.172-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19960000 2016-04-07T03:49:14.221-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19970000 2016-04-07T03:49:14.264-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19980000 2016-04-07T03:49:14.315-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 19990000 2016-04-07T03:49:14.315-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13320000 QueuedCount 50000 2016-04-07T03:49:14.755-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13330000 QueuedCount 50000 2016-04-07T03:49:14.932-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:15.142-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13340000 QueuedCount 50000 2016-04-07T03:49:15.509-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13350000 QueuedCount 50000 2016-04-07T03:49:15.926-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13360000 QueuedCount 50000 2016-04-07T03:49:16.336-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13370000 QueuedCount 50000 2016-04-07T03:49:16.712-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:16.72-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13380000 QueuedCount 49219 2016-04-07T03:49:17.127-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13390000 QueuedCount 50000 2016-04-07T03:49:17.559-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13400000 QueuedCount 50000 2016-04-07T03:49:17.954-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13410000 QueuedCount 50000 2016-04-07T03:49:18.368-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13420000 QueuedCount 50000 2016-04-07T03:49:18.763-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13430000 QueuedCount 40637 2016-04-07T03:49:19.099-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13440000 QueuedCount 30637 2016-04-07T03:49:19.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 199 2016-04-07T03:49:19.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 199 2016-04-07T03:49:19.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 199 2016-04-07T03:49:19.456-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13450000 QueuedCount 20637 2016-04-07T03:49:19.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 199 2016-04-07T03:49:19.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 199 2016-04-07T03:49:19.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 199 2016-04-07T03:49:19.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 199 2016-04-07T03:49:19.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 199 2016-04-07T03:49:19.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 199 2016-04-07T03:49:19.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 199 2016-04-07T03:49:19.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 199 2016-04-07T03:49:19.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 199 2016-04-07T03:49:19.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 199 2016-04-07T03:49:19.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 199 2016-04-07T03:49:19.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 199 2016-04-07T03:49:19.805-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13460000 QueuedCount 10637 2016-04-07T03:49:19.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 199 2016-04-07T03:49:19.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 199 2016-04-07T03:49:19.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 199 2016-04-07T03:49:19.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 199 2016-04-07T03:49:19.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 199 2016-04-07T03:49:19.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 199 2016-04-07T03:49:20-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 199 2016-04-07T03:49:20.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 199 2016-04-07T03:49:20.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 199 2016-04-07T03:49:20.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 199 2016-04-07T03:49:20.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 199 2016-04-07T03:49:20.148-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13470000 QueuedCount 637 2016-04-07T03:49:20.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 199 2016-04-07T03:49:20.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 199 2016-04-07T03:49:20.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 199 2016-04-07T03:49:20.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 199 2016-04-07T03:49:20.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 199 2016-04-07T03:49:20.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 199 2016-04-07T03:49:20.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 199 2016-04-07T03:49:20.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 199 2016-04-07T03:49:20.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 199 2016-04-07T03:49:20.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 199 2016-04-07T03:49:20.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 199 2016-04-07T03:49:20.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 199 2016-04-07T03:49:20.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 199 2016-04-07T03:49:20.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 199 2016-04-07T03:49:20.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 199 2016-04-07T03:49:20.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 199 2016-04-07T03:49:20.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 199 2016-04-07T03:49:20.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 199 2016-04-07T03:49:20.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 199 2016-04-07T03:49:20.715-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.626µs CommitTime 547.067359ms TotalFlushTime 8m52.476263616s TotalCommitTime 34.5584431s 2016-04-07T03:49:20.715-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 13470637, 13470637, 98 committed:true 2016-04-07T03:49:20.715-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 13470637, 13470637, 98 committed:true) SnapCreateDur 557.026652ms SnapOpenDur 36.32µs 2016-04-07T03:49:20.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 199 2016-04-07T03:49:20.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 199 2016-04-07T03:49:20.76-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:20.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 199 2016-04-07T03:49:20.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 199 2016-04-07T03:49:20.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 199 2016-04-07T03:49:20.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 199 2016-04-07T03:49:20.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 199 2016-04-07T03:49:20.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 199 2016-04-07T03:49:20.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 199 2016-04-07T03:49:20.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 199 2016-04-07T03:49:21.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 199 2016-04-07T03:49:21.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 199 2016-04-07T03:49:21.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 199 2016-04-07T03:49:21.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 199 2016-04-07T03:49:21.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 199 2016-04-07T03:49:21.142-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13480000 QueuedCount 50000 2016-04-07T03:49:21.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 199 2016-04-07T03:49:21.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 199 2016-04-07T03:49:21.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 199 2016-04-07T03:49:21.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 199 2016-04-07T03:49:21.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 199 2016-04-07T03:49:21.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 199 2016-04-07T03:49:21.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 199 2016-04-07T03:49:21.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 199 2016-04-07T03:49:21.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 199 2016-04-07T03:49:21.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 199 2016-04-07T03:49:21.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 199 2016-04-07T03:49:21.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 199 2016-04-07T03:49:21.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 199 2016-04-07T03:49:21.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 199 2016-04-07T03:49:21.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 199 2016-04-07T03:49:21.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 199 2016-04-07T03:49:21.629-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13490000 QueuedCount 50000 2016-04-07T03:49:21.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 199 2016-04-07T03:49:21.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 199 2016-04-07T03:49:21.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 199 2016-04-07T03:49:21.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 199 2016-04-07T03:49:21.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 199 2016-04-07T03:49:21.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 199 2016-04-07T03:49:21.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 199 2016-04-07T03:49:21.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 199 2016-04-07T03:49:21.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 199 2016-04-07T03:49:21.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 199 2016-04-07T03:49:21.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 199 2016-04-07T03:49:21.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 199 2016-04-07T03:49:22.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 199 2016-04-07T03:49:22.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 199 2016-04-07T03:49:22.05-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13500000 QueuedCount 50000 2016-04-07T03:49:22.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 199 2016-04-07T03:49:22.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 199 2016-04-07T03:49:22.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 199 2016-04-07T03:49:22.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 199 2016-04-07T03:49:22.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 199 2016-04-07T03:49:22.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 199 2016-04-07T03:49:22.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 199 2016-04-07T03:49:22.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 199 2016-04-07T03:49:22.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 199 2016-04-07T03:49:22.315-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:22.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 199 2016-04-07T03:49:22.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 199 2016-04-07T03:49:22.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 199 2016-04-07T03:49:22.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 199 2016-04-07T03:49:22.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 199 2016-04-07T03:49:22.48-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13510000 QueuedCount 50000 2016-04-07T03:49:22.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 199 2016-04-07T03:49:22.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 199 2016-04-07T03:49:22.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 199 2016-04-07T03:49:22.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 199 2016-04-07T03:49:22.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 199 2016-04-07T03:49:22.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 199 2016-04-07T03:49:22.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 199 2016-04-07T03:49:22.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 199 2016-04-07T03:49:22.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 199 2016-04-07T03:49:22.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 199 2016-04-07T03:49:22.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 199 2016-04-07T03:49:22.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 199 2016-04-07T03:49:22.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 199 2016-04-07T03:49:22.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 199 2016-04-07T03:49:22.9-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13520000 QueuedCount 50000 2016-04-07T03:49:22.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 199 2016-04-07T03:49:22.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 199 2016-04-07T03:49:22.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 199 2016-04-07T03:49:23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 199 2016-04-07T03:49:23.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 199 2016-04-07T03:49:23.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8700 Milliseconds Vbucket 199 2016-04-07T03:49:23.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8730 Milliseconds Vbucket 199 2016-04-07T03:49:23.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8760 Milliseconds Vbucket 199 2016-04-07T03:49:23.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8790 Milliseconds Vbucket 199 2016-04-07T03:49:23.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8820 Milliseconds Vbucket 199 2016-04-07T03:49:23.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8850 Milliseconds Vbucket 199 2016-04-07T03:49:23.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8880 Milliseconds Vbucket 199 2016-04-07T03:49:23.256-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13530000 QueuedCount 50000 2016-04-07T03:49:23.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8910 Milliseconds Vbucket 199 2016-04-07T03:49:23.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8940 Milliseconds Vbucket 199 2016-04-07T03:49:23.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8970 Milliseconds Vbucket 199 2016-04-07T03:49:23.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9000 Milliseconds Vbucket 199 2016-04-07T03:49:23.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9030 Milliseconds Vbucket 199 2016-04-07T03:49:23.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9060 Milliseconds Vbucket 199 2016-04-07T03:49:23.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9090 Milliseconds Vbucket 199 2016-04-07T03:49:23.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9120 Milliseconds Vbucket 199 2016-04-07T03:49:23.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9150 Milliseconds Vbucket 199 2016-04-07T03:49:23.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9180 Milliseconds Vbucket 199 2016-04-07T03:49:23.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9210 Milliseconds Vbucket 199 2016-04-07T03:49:23.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9240 Milliseconds Vbucket 199 2016-04-07T03:49:23.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9270 Milliseconds Vbucket 199 2016-04-07T03:49:23.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9300 Milliseconds Vbucket 199 2016-04-07T03:49:23.687-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13540000 QueuedCount 50000 2016-04-07T03:49:23.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9330 Milliseconds Vbucket 199 2016-04-07T03:49:23.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9360 Milliseconds Vbucket 199 2016-04-07T03:49:23.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9390 Milliseconds Vbucket 199 2016-04-07T03:49:23.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9420 Milliseconds Vbucket 199 2016-04-07T03:49:23.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9450 Milliseconds Vbucket 199 2016-04-07T03:49:23.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9480 Milliseconds Vbucket 199 2016-04-07T03:49:23.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9510 Milliseconds Vbucket 199 2016-04-07T03:49:23.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9540 Milliseconds Vbucket 199 2016-04-07T03:49:23.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9570 Milliseconds Vbucket 199 2016-04-07T03:49:23.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9600 Milliseconds Vbucket 199 2016-04-07T03:49:23.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9630 Milliseconds Vbucket 199 2016-04-07T03:49:24.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9660 Milliseconds Vbucket 199 2016-04-07T03:49:24.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9690 Milliseconds Vbucket 199 2016-04-07T03:49:24.06-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13550000 QueuedCount 43146 2016-04-07T03:49:24.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9720 Milliseconds Vbucket 199 2016-04-07T03:49:24.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9750 Milliseconds Vbucket 199 2016-04-07T03:49:24.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9780 Milliseconds Vbucket 199 2016-04-07T03:49:24.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9810 Milliseconds Vbucket 199 2016-04-07T03:49:24.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9840 Milliseconds Vbucket 199 2016-04-07T03:49:24.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9870 Milliseconds Vbucket 199 2016-04-07T03:49:24.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9900 Milliseconds Vbucket 199 2016-04-07T03:49:24.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9930 Milliseconds Vbucket 199 2016-04-07T03:49:24.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9960 Milliseconds Vbucket 199 2016-04-07T03:49:24.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9990 Milliseconds Vbucket 199 2016-04-07T03:49:24.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10020 Milliseconds Vbucket 199 2016-04-07T03:49:24.408-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13560000 QueuedCount 33146 2016-04-07T03:49:24.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10050 Milliseconds Vbucket 199 2016-04-07T03:49:24.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10080 Milliseconds Vbucket 199 2016-04-07T03:49:24.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10110 Milliseconds Vbucket 199 2016-04-07T03:49:24.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10140 Milliseconds Vbucket 199 2016-04-07T03:49:24.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10170 Milliseconds Vbucket 199 2016-04-07T03:49:24.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10200 Milliseconds Vbucket 199 2016-04-07T03:49:24.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10230 Milliseconds Vbucket 199 2016-04-07T03:49:24.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10260 Milliseconds Vbucket 199 2016-04-07T03:49:24.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10290 Milliseconds Vbucket 199 2016-04-07T03:49:24.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10320 Milliseconds Vbucket 199 2016-04-07T03:49:24.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10350 Milliseconds Vbucket 199 2016-04-07T03:49:24.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10380 Milliseconds Vbucket 199 2016-04-07T03:49:24.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10410 Milliseconds Vbucket 199 2016-04-07T03:49:24.792-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13570000 QueuedCount 23146 2016-04-07T03:49:24.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10440 Milliseconds Vbucket 199 2016-04-07T03:49:24.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10470 Milliseconds Vbucket 199 2016-04-07T03:49:24.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10500 Milliseconds Vbucket 199 2016-04-07T03:49:24.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10530 Milliseconds Vbucket 199 2016-04-07T03:49:24.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10560 Milliseconds Vbucket 199 2016-04-07T03:49:24.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10590 Milliseconds Vbucket 199 2016-04-07T03:49:24.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10620 Milliseconds Vbucket 199 2016-04-07T03:49:25.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10650 Milliseconds Vbucket 199 2016-04-07T03:49:25.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10680 Milliseconds Vbucket 199 2016-04-07T03:49:25.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10710 Milliseconds Vbucket 199 2016-04-07T03:49:25.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10740 Milliseconds Vbucket 199 2016-04-07T03:49:25.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10770 Milliseconds Vbucket 199 2016-04-07T03:49:25.142-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13580000 QueuedCount 13146 2016-04-07T03:49:25.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10800 Milliseconds Vbucket 199 2016-04-07T03:49:25.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10830 Milliseconds Vbucket 199 2016-04-07T03:49:25.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10860 Milliseconds Vbucket 199 2016-04-07T03:49:25.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10890 Milliseconds Vbucket 199 2016-04-07T03:49:25.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10920 Milliseconds Vbucket 199 2016-04-07T03:49:25.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10950 Milliseconds Vbucket 199 2016-04-07T03:49:25.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10980 Milliseconds Vbucket 199 2016-04-07T03:49:25.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11010 Milliseconds Vbucket 199 2016-04-07T03:49:25.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11040 Milliseconds Vbucket 199 2016-04-07T03:49:25.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11070 Milliseconds Vbucket 199 2016-04-07T03:49:25.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11100 Milliseconds Vbucket 199 2016-04-07T03:49:25.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11130 Milliseconds Vbucket 199 2016-04-07T03:49:25.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11160 Milliseconds Vbucket 199 2016-04-07T03:49:25.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11190 Milliseconds Vbucket 199 2016-04-07T03:49:25.557-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13590000 QueuedCount 3146 2016-04-07T03:49:25.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11220 Milliseconds Vbucket 199 2016-04-07T03:49:25.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11250 Milliseconds Vbucket 199 2016-04-07T03:49:25.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11280 Milliseconds Vbucket 199 2016-04-07T03:49:25.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11310 Milliseconds Vbucket 199 2016-04-07T03:49:25.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11340 Milliseconds Vbucket 199 2016-04-07T03:49:25.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11370 Milliseconds Vbucket 199 2016-04-07T03:49:25.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11400 Milliseconds Vbucket 199 2016-04-07T03:49:25.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11430 Milliseconds Vbucket 199 2016-04-07T03:49:25.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11460 Milliseconds Vbucket 199 2016-04-07T03:49:25.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11490 Milliseconds Vbucket 199 2016-04-07T03:49:25.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11520 Milliseconds Vbucket 199 2016-04-07T03:49:25.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11550 Milliseconds Vbucket 199 2016-04-07T03:49:25.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11580 Milliseconds Vbucket 199 2016-04-07T03:49:25.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11610 Milliseconds Vbucket 199 2016-04-07T03:49:26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11640 Milliseconds Vbucket 199 2016-04-07T03:49:26.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11670 Milliseconds Vbucket 199 2016-04-07T03:49:26.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11700 Milliseconds Vbucket 199 2016-04-07T03:49:26.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11730 Milliseconds Vbucket 199 2016-04-07T03:49:26.115-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.191µs CommitTime 458.600913ms TotalFlushTime 8m57.213275414s TotalCommitTime 35.017044013s 2016-04-07T03:49:26.115-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 13593146, 13593146, 99 committed:true 2016-04-07T03:49:26.115-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 13593146, 13593146, 99 committed:true) SnapCreateDur 468.954561ms SnapOpenDur 36.599µs 2016-04-07T03:49:26.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11760 Milliseconds Vbucket 199 2016-04-07T03:49:26.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11790 Milliseconds Vbucket 199 2016-04-07T03:49:26.154-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:26.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11820 Milliseconds Vbucket 199 2016-04-07T03:49:26.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11850 Milliseconds Vbucket 199 2016-04-07T03:49:26.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11880 Milliseconds Vbucket 199 2016-04-07T03:49:26.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11910 Milliseconds Vbucket 199 2016-04-07T03:49:26.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11940 Milliseconds Vbucket 199 2016-04-07T03:49:26.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11970 Milliseconds Vbucket 199 2016-04-07T03:49:26.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12000 Milliseconds Vbucket 199 2016-04-07T03:49:26.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12030 Milliseconds Vbucket 199 2016-04-07T03:49:26.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12060 Milliseconds Vbucket 199 2016-04-07T03:49:26.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12090 Milliseconds Vbucket 199 2016-04-07T03:49:26.469-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13600000 QueuedCount 50000 2016-04-07T03:49:26.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12120 Milliseconds Vbucket 199 2016-04-07T03:49:26.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12150 Milliseconds Vbucket 199 2016-04-07T03:49:26.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12180 Milliseconds Vbucket 199 2016-04-07T03:49:26.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12210 Milliseconds Vbucket 199 2016-04-07T03:49:26.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12240 Milliseconds Vbucket 199 2016-04-07T03:49:26.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12270 Milliseconds Vbucket 199 2016-04-07T03:49:26.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12300 Milliseconds Vbucket 199 2016-04-07T03:49:26.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12330 Milliseconds Vbucket 199 2016-04-07T03:49:26.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12360 Milliseconds Vbucket 199 2016-04-07T03:49:26.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12390 Milliseconds Vbucket 199 2016-04-07T03:49:26.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12420 Milliseconds Vbucket 199 2016-04-07T03:49:26.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12450 Milliseconds Vbucket 199 2016-04-07T03:49:26.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12480 Milliseconds Vbucket 199 2016-04-07T03:49:26.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12510 Milliseconds Vbucket 199 2016-04-07T03:49:26.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12540 Milliseconds Vbucket 199 2016-04-07T03:49:26.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12570 Milliseconds Vbucket 199 2016-04-07T03:49:26.936-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13610000 QueuedCount 50000 2016-04-07T03:49:26.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12600 Milliseconds Vbucket 199 2016-04-07T03:49:26.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12630 Milliseconds Vbucket 199 2016-04-07T03:49:27.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12660 Milliseconds Vbucket 199 2016-04-07T03:49:27.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12690 Milliseconds Vbucket 199 2016-04-07T03:49:27.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12720 Milliseconds Vbucket 199 2016-04-07T03:49:27.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12750 Milliseconds Vbucket 199 2016-04-07T03:49:27.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12780 Milliseconds Vbucket 199 2016-04-07T03:49:27.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12810 Milliseconds Vbucket 199 2016-04-07T03:49:27.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12840 Milliseconds Vbucket 199 2016-04-07T03:49:27.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12870 Milliseconds Vbucket 199 2016-04-07T03:49:27.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12900 Milliseconds Vbucket 199 2016-04-07T03:49:27.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12930 Milliseconds Vbucket 199 2016-04-07T03:49:27.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12960 Milliseconds Vbucket 199 2016-04-07T03:49:27.338-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:27.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 12990 Milliseconds Vbucket 199 2016-04-07T03:49:27.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13020 Milliseconds Vbucket 199 2016-04-07T03:49:27.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13050 Milliseconds Vbucket 199 2016-04-07T03:49:27.429-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13620000 QueuedCount 50000 2016-04-07T03:49:27.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13080 Milliseconds Vbucket 199 2016-04-07T03:49:27.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13110 Milliseconds Vbucket 199 2016-04-07T03:49:27.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13140 Milliseconds Vbucket 199 2016-04-07T03:49:27.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13170 Milliseconds Vbucket 199 2016-04-07T03:49:27.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13200 Milliseconds Vbucket 199 2016-04-07T03:49:27.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13230 Milliseconds Vbucket 199 2016-04-07T03:49:27.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13260 Milliseconds Vbucket 199 2016-04-07T03:49:27.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13290 Milliseconds Vbucket 199 2016-04-07T03:49:27.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13320 Milliseconds Vbucket 199 2016-04-07T03:49:27.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13350 Milliseconds Vbucket 199 2016-04-07T03:49:27.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13380 Milliseconds Vbucket 199 2016-04-07T03:49:27.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13410 Milliseconds Vbucket 199 2016-04-07T03:49:27.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13440 Milliseconds Vbucket 199 2016-04-07T03:49:27.818-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13630000 QueuedCount 50000 2016-04-07T03:49:27.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13470 Milliseconds Vbucket 199 2016-04-07T03:49:27.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13500 Milliseconds Vbucket 199 2016-04-07T03:49:27.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13530 Milliseconds Vbucket 199 2016-04-07T03:49:27.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13560 Milliseconds Vbucket 199 2016-04-07T03:49:27.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13590 Milliseconds Vbucket 199 2016-04-07T03:49:27.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13620 Milliseconds Vbucket 199 2016-04-07T03:49:28.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13650 Milliseconds Vbucket 199 2016-04-07T03:49:28.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13680 Milliseconds Vbucket 199 2016-04-07T03:49:28.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13710 Milliseconds Vbucket 199 2016-04-07T03:49:28.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13740 Milliseconds Vbucket 199 2016-04-07T03:49:28.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13770 Milliseconds Vbucket 199 2016-04-07T03:49:28.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13800 Milliseconds Vbucket 199 2016-04-07T03:49:28.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13830 Milliseconds Vbucket 199 2016-04-07T03:49:28.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13860 Milliseconds Vbucket 199 2016-04-07T03:49:28.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13890 Milliseconds Vbucket 199 2016-04-07T03:49:28.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13920 Milliseconds Vbucket 199 2016-04-07T03:49:28.303-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13640000 QueuedCount 50000 2016-04-07T03:49:28.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13950 Milliseconds Vbucket 199 2016-04-07T03:49:28.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 13980 Milliseconds Vbucket 199 2016-04-07T03:49:28.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14010 Milliseconds Vbucket 199 2016-04-07T03:49:28.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14040 Milliseconds Vbucket 199 2016-04-07T03:49:28.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14070 Milliseconds Vbucket 199 2016-04-07T03:49:28.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14100 Milliseconds Vbucket 199 2016-04-07T03:49:28.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14130 Milliseconds Vbucket 199 2016-04-07T03:49:28.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14160 Milliseconds Vbucket 199 2016-04-07T03:49:28.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14190 Milliseconds Vbucket 199 2016-04-07T03:49:28.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14220 Milliseconds Vbucket 199 2016-04-07T03:49:28.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14250 Milliseconds Vbucket 199 2016-04-07T03:49:28.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14280 Milliseconds Vbucket 199 2016-04-07T03:49:28.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14310 Milliseconds Vbucket 199 2016-04-07T03:49:28.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14340 Milliseconds Vbucket 199 2016-04-07T03:49:28.722-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13650000 QueuedCount 50000 2016-04-07T03:49:28.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14370 Milliseconds Vbucket 199 2016-04-07T03:49:28.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14400 Milliseconds Vbucket 199 2016-04-07T03:49:28.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14430 Milliseconds Vbucket 199 2016-04-07T03:49:28.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14460 Milliseconds Vbucket 199 2016-04-07T03:49:28.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14490 Milliseconds Vbucket 199 2016-04-07T03:49:28.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14520 Milliseconds Vbucket 199 2016-04-07T03:49:28.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14550 Milliseconds Vbucket 199 2016-04-07T03:49:28.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14580 Milliseconds Vbucket 199 2016-04-07T03:49:28.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14610 Milliseconds Vbucket 199 2016-04-07T03:49:29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14640 Milliseconds Vbucket 199 2016-04-07T03:49:29.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14670 Milliseconds Vbucket 199 2016-04-07T03:49:29.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14700 Milliseconds Vbucket 199 2016-04-07T03:49:29.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14730 Milliseconds Vbucket 199 2016-04-07T03:49:29.116-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13660000 QueuedCount 45110 2016-04-07T03:49:29.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14760 Milliseconds Vbucket 199 2016-04-07T03:49:29.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14790 Milliseconds Vbucket 199 2016-04-07T03:49:29.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14820 Milliseconds Vbucket 199 2016-04-07T03:49:29.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14850 Milliseconds Vbucket 199 2016-04-07T03:49:29.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14880 Milliseconds Vbucket 199 2016-04-07T03:49:29.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14910 Milliseconds Vbucket 199 2016-04-07T03:49:29.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14940 Milliseconds Vbucket 199 2016-04-07T03:49:29.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 14970 Milliseconds Vbucket 199 2016-04-07T03:49:29.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15000 Milliseconds Vbucket 199 2016-04-07T03:49:29.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15030 Milliseconds Vbucket 199 2016-04-07T03:49:29.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15060 Milliseconds Vbucket 199 2016-04-07T03:49:29.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15090 Milliseconds Vbucket 199 2016-04-07T03:49:29.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15120 Milliseconds Vbucket 199 2016-04-07T03:49:29.492-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13670000 QueuedCount 35110 2016-04-07T03:49:29.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15150 Milliseconds Vbucket 199 2016-04-07T03:49:29.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15180 Milliseconds Vbucket 199 2016-04-07T03:49:29.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15210 Milliseconds Vbucket 199 2016-04-07T03:49:29.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15240 Milliseconds Vbucket 199 2016-04-07T03:49:29.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15270 Milliseconds Vbucket 199 2016-04-07T03:49:29.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15300 Milliseconds Vbucket 199 2016-04-07T03:49:29.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15330 Milliseconds Vbucket 199 2016-04-07T03:49:29.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15360 Milliseconds Vbucket 199 2016-04-07T03:49:29.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15390 Milliseconds Vbucket 199 2016-04-07T03:49:29.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15420 Milliseconds Vbucket 199 2016-04-07T03:49:29.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15450 Milliseconds Vbucket 199 2016-04-07T03:49:29.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15480 Milliseconds Vbucket 199 2016-04-07T03:49:29.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15510 Milliseconds Vbucket 199 2016-04-07T03:49:29.887-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13680000 QueuedCount 25110 2016-04-07T03:49:29.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15540 Milliseconds Vbucket 199 2016-04-07T03:49:29.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15570 Milliseconds Vbucket 199 2016-04-07T03:49:29.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15600 Milliseconds Vbucket 199 2016-04-07T03:49:29.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15630 Milliseconds Vbucket 199 2016-04-07T03:49:30.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15660 Milliseconds Vbucket 199 2016-04-07T03:49:30.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15690 Milliseconds Vbucket 199 2016-04-07T03:49:30.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15720 Milliseconds Vbucket 199 2016-04-07T03:49:30.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15750 Milliseconds Vbucket 199 2016-04-07T03:49:30.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15780 Milliseconds Vbucket 199 2016-04-07T03:49:30.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15810 Milliseconds Vbucket 199 2016-04-07T03:49:30.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15840 Milliseconds Vbucket 199 2016-04-07T03:49:30.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15870 Milliseconds Vbucket 199 2016-04-07T03:49:30.244-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13690000 QueuedCount 15110 2016-04-07T03:49:30.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15900 Milliseconds Vbucket 199 2016-04-07T03:49:30.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15930 Milliseconds Vbucket 199 2016-04-07T03:49:30.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15960 Milliseconds Vbucket 199 2016-04-07T03:49:30.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 15990 Milliseconds Vbucket 199 2016-04-07T03:49:30.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16020 Milliseconds Vbucket 199 2016-04-07T03:49:30.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16050 Milliseconds Vbucket 199 2016-04-07T03:49:30.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16080 Milliseconds Vbucket 199 2016-04-07T03:49:30.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16110 Milliseconds Vbucket 199 2016-04-07T03:49:30.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16140 Milliseconds Vbucket 199 2016-04-07T03:49:30.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16170 Milliseconds Vbucket 199 2016-04-07T03:49:30.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16200 Milliseconds Vbucket 199 2016-04-07T03:49:30.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16230 Milliseconds Vbucket 199 2016-04-07T03:49:30.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16260 Milliseconds Vbucket 199 2016-04-07T03:49:30.633-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13700000 QueuedCount 5110 2016-04-07T03:49:30.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16290 Milliseconds Vbucket 199 2016-04-07T03:49:30.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16320 Milliseconds Vbucket 199 2016-04-07T03:49:30.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16350 Milliseconds Vbucket 199 2016-04-07T03:49:30.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16380 Milliseconds Vbucket 199 2016-04-07T03:49:30.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16410 Milliseconds Vbucket 199 2016-04-07T03:49:30.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16440 Milliseconds Vbucket 199 2016-04-07T03:49:30.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16470 Milliseconds Vbucket 199 2016-04-07T03:49:30.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16500 Milliseconds Vbucket 199 2016-04-07T03:49:30.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16530 Milliseconds Vbucket 199 2016-04-07T03:49:30.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16560 Milliseconds Vbucket 199 2016-04-07T03:49:30.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16590 Milliseconds Vbucket 199 2016-04-07T03:49:30.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16620 Milliseconds Vbucket 199 2016-04-07T03:49:31.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16650 Milliseconds Vbucket 199 2016-04-07T03:49:31.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16680 Milliseconds Vbucket 199 2016-04-07T03:49:31.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16710 Milliseconds Vbucket 199 2016-04-07T03:49:31.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16740 Milliseconds Vbucket 199 2016-04-07T03:49:31.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16770 Milliseconds Vbucket 199 2016-04-07T03:49:31.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16800 Milliseconds Vbucket 199 2016-04-07T03:49:31.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16830 Milliseconds Vbucket 199 2016-04-07T03:49:31.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16860 Milliseconds Vbucket 199 2016-04-07T03:49:31.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16890 Milliseconds Vbucket 199 2016-04-07T03:49:31.271-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.008µs CommitTime 458.913569ms TotalFlushTime 9m1.713383649s TotalCommitTime 35.475957582s 2016-04-07T03:49:31.271-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 13705110, 13705110, 100 committed:true 2016-04-07T03:49:31.271-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 13705110, 13705110, 100 committed:true) SnapCreateDur 469.976963ms SnapOpenDur 41.493µs 2016-04-07T03:49:31.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16920 Milliseconds Vbucket 199 2016-04-07T03:49:31.294-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:31.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16950 Milliseconds Vbucket 199 2016-04-07T03:49:31.331-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:31.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 16980 Milliseconds Vbucket 199 2016-04-07T03:49:31.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17010 Milliseconds Vbucket 199 2016-04-07T03:49:31.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17040 Milliseconds Vbucket 199 2016-04-07T03:49:31.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17070 Milliseconds Vbucket 199 2016-04-07T03:49:31.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17100 Milliseconds Vbucket 199 2016-04-07T03:49:31.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17130 Milliseconds Vbucket 199 2016-04-07T03:49:31.499-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13710000 QueuedCount 50000 2016-04-07T03:49:31.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17160 Milliseconds Vbucket 199 2016-04-07T03:49:31.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17190 Milliseconds Vbucket 199 2016-04-07T03:49:31.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17220 Milliseconds Vbucket 199 2016-04-07T03:49:31.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17250 Milliseconds Vbucket 199 2016-04-07T03:49:31.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17280 Milliseconds Vbucket 199 2016-04-07T03:49:31.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17310 Milliseconds Vbucket 199 2016-04-07T03:49:31.697-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:31.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17340 Milliseconds Vbucket 199 2016-04-07T03:49:31.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17370 Milliseconds Vbucket 199 2016-04-07T03:49:31.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17400 Milliseconds Vbucket 199 2016-04-07T03:49:31.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17430 Milliseconds Vbucket 199 2016-04-07T03:49:31.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17460 Milliseconds Vbucket 199 2016-04-07T03:49:31.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17490 Milliseconds Vbucket 199 2016-04-07T03:49:31.87-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:31.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17520 Milliseconds Vbucket 199 2016-04-07T03:49:31.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17550 Milliseconds Vbucket 199 2016-04-07T03:49:31.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17580 Milliseconds Vbucket 199 2016-04-07T03:49:31.951-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13720000 QueuedCount 50000 2016-04-07T03:49:31.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17610 Milliseconds Vbucket 199 2016-04-07T03:49:32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17640 Milliseconds Vbucket 199 2016-04-07T03:49:32.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17670 Milliseconds Vbucket 199 2016-04-07T03:49:32.049-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:32.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17700 Milliseconds Vbucket 199 2016-04-07T03:49:32.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17730 Milliseconds Vbucket 199 2016-04-07T03:49:32.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17760 Milliseconds Vbucket 199 2016-04-07T03:49:32.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17790 Milliseconds Vbucket 199 2016-04-07T03:49:32.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17820 Milliseconds Vbucket 199 2016-04-07T03:49:32.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17850 Milliseconds Vbucket 199 2016-04-07T03:49:32.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17880 Milliseconds Vbucket 199 2016-04-07T03:49:32.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17910 Milliseconds Vbucket 199 2016-04-07T03:49:32.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17940 Milliseconds Vbucket 199 2016-04-07T03:49:32.317-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:32.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 17970 Milliseconds Vbucket 199 2016-04-07T03:49:32.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18000 Milliseconds Vbucket 199 2016-04-07T03:49:32.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18030 Milliseconds Vbucket 199 2016-04-07T03:49:32.407-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13730000 QueuedCount 50000 2016-04-07T03:49:32.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18060 Milliseconds Vbucket 199 2016-04-07T03:49:32.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18090 Milliseconds Vbucket 199 2016-04-07T03:49:32.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18120 Milliseconds Vbucket 199 2016-04-07T03:49:32.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18150 Milliseconds Vbucket 199 2016-04-07T03:49:32.516-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:32.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18180 Milliseconds Vbucket 199 2016-04-07T03:49:32.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18210 Milliseconds Vbucket 199 2016-04-07T03:49:32.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18240 Milliseconds Vbucket 199 2016-04-07T03:49:32.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18270 Milliseconds Vbucket 199 2016-04-07T03:49:32.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18300 Milliseconds Vbucket 199 2016-04-07T03:49:32.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18330 Milliseconds Vbucket 199 2016-04-07T03:49:32.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18360 Milliseconds Vbucket 199 2016-04-07T03:49:32.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18390 Milliseconds Vbucket 199 2016-04-07T03:49:32.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18420 Milliseconds Vbucket 199 2016-04-07T03:49:32.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18450 Milliseconds Vbucket 199 2016-04-07T03:49:32.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18480 Milliseconds Vbucket 199 2016-04-07T03:49:32.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18510 Milliseconds Vbucket 199 2016-04-07T03:49:32.873-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13740000 QueuedCount 47839 2016-04-07T03:49:32.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18540 Milliseconds Vbucket 199 2016-04-07T03:49:32.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18570 Milliseconds Vbucket 199 2016-04-07T03:49:32.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18600 Milliseconds Vbucket 199 2016-04-07T03:49:32.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18630 Milliseconds Vbucket 199 2016-04-07T03:49:33.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18660 Milliseconds Vbucket 199 2016-04-07T03:49:33.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18690 Milliseconds Vbucket 199 2016-04-07T03:49:33.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18720 Milliseconds Vbucket 199 2016-04-07T03:49:33.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18750 Milliseconds Vbucket 199 2016-04-07T03:49:33.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18780 Milliseconds Vbucket 199 2016-04-07T03:49:33.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18810 Milliseconds Vbucket 199 2016-04-07T03:49:33.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18840 Milliseconds Vbucket 199 2016-04-07T03:49:33.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18870 Milliseconds Vbucket 199 2016-04-07T03:49:33.259-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13750000 QueuedCount 37839 2016-04-07T03:49:33.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18900 Milliseconds Vbucket 199 2016-04-07T03:49:33.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18930 Milliseconds Vbucket 199 2016-04-07T03:49:33.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18960 Milliseconds Vbucket 199 2016-04-07T03:49:33.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 18990 Milliseconds Vbucket 199 2016-04-07T03:49:33.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19020 Milliseconds Vbucket 199 2016-04-07T03:49:33.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19050 Milliseconds Vbucket 199 2016-04-07T03:49:33.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19080 Milliseconds Vbucket 199 2016-04-07T03:49:33.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19110 Milliseconds Vbucket 199 2016-04-07T03:49:33.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19140 Milliseconds Vbucket 199 2016-04-07T03:49:33.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19170 Milliseconds Vbucket 199 2016-04-07T03:49:33.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19200 Milliseconds Vbucket 199 2016-04-07T03:49:33.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19230 Milliseconds Vbucket 199 2016-04-07T03:49:33.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19260 Milliseconds Vbucket 199 2016-04-07T03:49:33.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19290 Milliseconds Vbucket 199 2016-04-07T03:49:33.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19320 Milliseconds Vbucket 199 2016-04-07T03:49:33.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19350 Milliseconds Vbucket 199 2016-04-07T03:49:33.71-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13760000 QueuedCount 27839 2016-04-07T03:49:33.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19380 Milliseconds Vbucket 199 2016-04-07T03:49:33.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19410 Milliseconds Vbucket 199 2016-04-07T03:49:33.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19440 Milliseconds Vbucket 199 2016-04-07T03:49:33.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19470 Milliseconds Vbucket 199 2016-04-07T03:49:33.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19500 Milliseconds Vbucket 199 2016-04-07T03:49:33.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19530 Milliseconds Vbucket 199 2016-04-07T03:49:33.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19560 Milliseconds Vbucket 199 2016-04-07T03:49:33.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19590 Milliseconds Vbucket 199 2016-04-07T03:49:33.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19620 Milliseconds Vbucket 199 2016-04-07T03:49:34.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19650 Milliseconds Vbucket 199 2016-04-07T03:49:34.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19680 Milliseconds Vbucket 199 2016-04-07T03:49:34.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19710 Milliseconds Vbucket 199 2016-04-07T03:49:34.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19740 Milliseconds Vbucket 199 2016-04-07T03:49:34.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19770 Milliseconds Vbucket 199 2016-04-07T03:49:34.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19800 Milliseconds Vbucket 199 2016-04-07T03:49:34.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19830 Milliseconds Vbucket 199 2016-04-07T03:49:34.216-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13770000 QueuedCount 17839 2016-04-07T03:49:34.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19860 Milliseconds Vbucket 199 2016-04-07T03:49:34.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19890 Milliseconds Vbucket 199 2016-04-07T03:49:34.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19920 Milliseconds Vbucket 199 2016-04-07T03:49:34.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19950 Milliseconds Vbucket 199 2016-04-07T03:49:34.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 19980 Milliseconds Vbucket 199 2016-04-07T03:49:34.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20010 Milliseconds Vbucket 199 2016-04-07T03:49:34.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20040 Milliseconds Vbucket 199 2016-04-07T03:49:34.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20070 Milliseconds Vbucket 199 2016-04-07T03:49:34.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20100 Milliseconds Vbucket 199 2016-04-07T03:49:34.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20130 Milliseconds Vbucket 199 2016-04-07T03:49:34.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20160 Milliseconds Vbucket 199 2016-04-07T03:49:34.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20190 Milliseconds Vbucket 199 2016-04-07T03:49:34.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20220 Milliseconds Vbucket 199 2016-04-07T03:49:34.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20250 Milliseconds Vbucket 199 2016-04-07T03:49:34.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20280 Milliseconds Vbucket 199 2016-04-07T03:49:34.658-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13780000 QueuedCount 7839 2016-04-07T03:49:34.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20310 Milliseconds Vbucket 199 2016-04-07T03:49:34.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20340 Milliseconds Vbucket 199 2016-04-07T03:49:34.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20370 Milliseconds Vbucket 199 2016-04-07T03:49:34.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20400 Milliseconds Vbucket 199 2016-04-07T03:49:34.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20430 Milliseconds Vbucket 199 2016-04-07T03:49:34.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20460 Milliseconds Vbucket 199 2016-04-07T03:49:34.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20490 Milliseconds Vbucket 199 2016-04-07T03:49:34.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20520 Milliseconds Vbucket 199 2016-04-07T03:49:34.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20550 Milliseconds Vbucket 199 2016-04-07T03:49:34.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20580 Milliseconds Vbucket 199 2016-04-07T03:49:34.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20610 Milliseconds Vbucket 199 2016-04-07T03:49:35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20640 Milliseconds Vbucket 199 2016-04-07T03:49:35.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20670 Milliseconds Vbucket 199 2016-04-07T03:49:35.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20700 Milliseconds Vbucket 199 2016-04-07T03:49:35.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20730 Milliseconds Vbucket 199 2016-04-07T03:49:35.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20760 Milliseconds Vbucket 199 2016-04-07T03:49:35.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20790 Milliseconds Vbucket 199 2016-04-07T03:49:35.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20820 Milliseconds Vbucket 199 2016-04-07T03:49:35.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20850 Milliseconds Vbucket 199 2016-04-07T03:49:35.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20880 Milliseconds Vbucket 199 2016-04-07T03:49:35.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20910 Milliseconds Vbucket 199 2016-04-07T03:49:35.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20940 Milliseconds Vbucket 199 2016-04-07T03:49:35.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 20970 Milliseconds Vbucket 199 2016-04-07T03:49:35.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21000 Milliseconds Vbucket 199 2016-04-07T03:49:35.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21030 Milliseconds Vbucket 199 2016-04-07T03:49:35.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21060 Milliseconds Vbucket 199 2016-04-07T03:49:35.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21090 Milliseconds Vbucket 199 2016-04-07T03:49:35.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21120 Milliseconds Vbucket 199 2016-04-07T03:49:35.481-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.504µs CommitTime 434.773131ms TotalFlushTime 9m5.371077829s TotalCommitTime 35.910730713s 2016-04-07T03:49:35.481-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 13787839, 13787839, 101 committed:true 2016-04-07T03:49:35.481-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 13787839, 13787839, 101 committed:true) SnapCreateDur 445.349825ms SnapOpenDur 62.768µs 2016-04-07T03:49:35.505-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:35.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21150 Milliseconds Vbucket 199 2016-04-07T03:49:35.527-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:35.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21180 Milliseconds Vbucket 199 2016-04-07T03:49:35.55-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:35.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21210 Milliseconds Vbucket 199 2016-04-07T03:49:35.572-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:35.594-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:35.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21240 Milliseconds Vbucket 199 2016-04-07T03:49:35.602-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13790000 QueuedCount 16768 2016-04-07T03:49:35.616-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:35.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21270 Milliseconds Vbucket 199 2016-04-07T03:49:35.638-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:35.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21300 Milliseconds Vbucket 199 2016-04-07T03:49:35.661-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:35.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21330 Milliseconds Vbucket 199 2016-04-07T03:49:35.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21360 Milliseconds Vbucket 199 2016-04-07T03:49:35.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21390 Milliseconds Vbucket 199 2016-04-07T03:49:35.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21420 Milliseconds Vbucket 199 2016-04-07T03:49:35.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21450 Milliseconds Vbucket 199 2016-04-07T03:49:35.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21480 Milliseconds Vbucket 199 2016-04-07T03:49:35.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21510 Milliseconds Vbucket 199 2016-04-07T03:49:35.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21540 Milliseconds Vbucket 199 2016-04-07T03:49:35.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21570 Milliseconds Vbucket 199 2016-04-07T03:49:35.936-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:35.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21600 Milliseconds Vbucket 199 2016-04-07T03:49:35.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21630 Milliseconds Vbucket 199 2016-04-07T03:49:36.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21660 Milliseconds Vbucket 199 2016-04-07T03:49:36.039-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13800000 QueuedCount 50000 2016-04-07T03:49:36.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21690 Milliseconds Vbucket 199 2016-04-07T03:49:36.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21720 Milliseconds Vbucket 199 2016-04-07T03:49:36.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21750 Milliseconds Vbucket 199 2016-04-07T03:49:36.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21780 Milliseconds Vbucket 199 2016-04-07T03:49:36.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21810 Milliseconds Vbucket 199 2016-04-07T03:49:36.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21840 Milliseconds Vbucket 199 2016-04-07T03:49:36.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21870 Milliseconds Vbucket 199 2016-04-07T03:49:36.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21900 Milliseconds Vbucket 199 2016-04-07T03:49:36.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21930 Milliseconds Vbucket 199 2016-04-07T03:49:36.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21960 Milliseconds Vbucket 199 2016-04-07T03:49:36.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 21990 Milliseconds Vbucket 199 2016-04-07T03:49:36.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22020 Milliseconds Vbucket 199 2016-04-07T03:49:36.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22050 Milliseconds Vbucket 199 2016-04-07T03:49:36.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22080 Milliseconds Vbucket 199 2016-04-07T03:49:36.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22110 Milliseconds Vbucket 199 2016-04-07T03:49:36.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22140 Milliseconds Vbucket 199 2016-04-07T03:49:36.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22170 Milliseconds Vbucket 199 2016-04-07T03:49:36.53-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13810000 QueuedCount 50000 2016-04-07T03:49:36.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22200 Milliseconds Vbucket 199 2016-04-07T03:49:36.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22230 Milliseconds Vbucket 199 2016-04-07T03:49:36.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22260 Milliseconds Vbucket 199 2016-04-07T03:49:36.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22290 Milliseconds Vbucket 199 2016-04-07T03:49:36.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22320 Milliseconds Vbucket 199 2016-04-07T03:49:36.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22350 Milliseconds Vbucket 199 2016-04-07T03:49:36.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22380 Milliseconds Vbucket 199 2016-04-07T03:49:36.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22410 Milliseconds Vbucket 199 2016-04-07T03:49:36.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22440 Milliseconds Vbucket 199 2016-04-07T03:49:36.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22470 Milliseconds Vbucket 199 2016-04-07T03:49:36.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22500 Milliseconds Vbucket 199 2016-04-07T03:49:36.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22530 Milliseconds Vbucket 199 2016-04-07T03:49:36.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22560 Milliseconds Vbucket 199 2016-04-07T03:49:36.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22590 Milliseconds Vbucket 199 2016-04-07T03:49:36.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22620 Milliseconds Vbucket 199 2016-04-07T03:49:36.985-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13820000 QueuedCount 50000 2016-04-07T03:49:37.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22650 Milliseconds Vbucket 199 2016-04-07T03:49:37.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22680 Milliseconds Vbucket 199 2016-04-07T03:49:37.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22710 Milliseconds Vbucket 199 2016-04-07T03:49:37.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22740 Milliseconds Vbucket 199 2016-04-07T03:49:37.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22770 Milliseconds Vbucket 199 2016-04-07T03:49:37.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22800 Milliseconds Vbucket 199 2016-04-07T03:49:37.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22830 Milliseconds Vbucket 199 2016-04-07T03:49:37.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22860 Milliseconds Vbucket 199 2016-04-07T03:49:37.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22890 Milliseconds Vbucket 199 2016-04-07T03:49:37.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22920 Milliseconds Vbucket 199 2016-04-07T03:49:37.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22950 Milliseconds Vbucket 199 2016-04-07T03:49:37.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 22980 Milliseconds Vbucket 199 2016-04-07T03:49:37.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23010 Milliseconds Vbucket 199 2016-04-07T03:49:37.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23040 Milliseconds Vbucket 199 2016-04-07T03:49:37.417-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13830000 QueuedCount 50000 2016-04-07T03:49:37.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23070 Milliseconds Vbucket 199 2016-04-07T03:49:37.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23100 Milliseconds Vbucket 199 2016-04-07T03:49:37.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23130 Milliseconds Vbucket 199 2016-04-07T03:49:37.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23160 Milliseconds Vbucket 199 2016-04-07T03:49:37.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23190 Milliseconds Vbucket 199 2016-04-07T03:49:37.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23220 Milliseconds Vbucket 199 2016-04-07T03:49:37.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23250 Milliseconds Vbucket 199 2016-04-07T03:49:37.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23280 Milliseconds Vbucket 199 2016-04-07T03:49:37.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23310 Milliseconds Vbucket 199 2016-04-07T03:49:37.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23340 Milliseconds Vbucket 199 2016-04-07T03:49:37.7-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:37.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23370 Milliseconds Vbucket 199 2016-04-07T03:49:37.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23400 Milliseconds Vbucket 199 2016-04-07T03:49:37.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23430 Milliseconds Vbucket 199 2016-04-07T03:49:37.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23460 Milliseconds Vbucket 199 2016-04-07T03:49:37.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23490 Milliseconds Vbucket 199 2016-04-07T03:49:37.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23520 Milliseconds Vbucket 199 2016-04-07T03:49:37.885-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13840000 QueuedCount 50000 2016-04-07T03:49:37.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23550 Milliseconds Vbucket 199 2016-04-07T03:49:37.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23580 Milliseconds Vbucket 199 2016-04-07T03:49:37.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23610 Milliseconds Vbucket 199 2016-04-07T03:49:38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23640 Milliseconds Vbucket 199 2016-04-07T03:49:38.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23670 Milliseconds Vbucket 199 2016-04-07T03:49:38.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23700 Milliseconds Vbucket 199 2016-04-07T03:49:38.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23730 Milliseconds Vbucket 199 2016-04-07T03:49:38.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23760 Milliseconds Vbucket 199 2016-04-07T03:49:38.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23790 Milliseconds Vbucket 199 2016-04-07T03:49:38.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23820 Milliseconds Vbucket 199 2016-04-07T03:49:38.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23850 Milliseconds Vbucket 199 2016-04-07T03:49:38.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23880 Milliseconds Vbucket 199 2016-04-07T03:49:38.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23910 Milliseconds Vbucket 199 2016-04-07T03:49:38.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23940 Milliseconds Vbucket 199 2016-04-07T03:49:38.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 23970 Milliseconds Vbucket 199 2016-04-07T03:49:38.35-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13850000 QueuedCount 50000 2016-04-07T03:49:38.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24000 Milliseconds Vbucket 199 2016-04-07T03:49:38.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24030 Milliseconds Vbucket 199 2016-04-07T03:49:38.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24060 Milliseconds Vbucket 199 2016-04-07T03:49:38.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24090 Milliseconds Vbucket 199 2016-04-07T03:49:38.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24120 Milliseconds Vbucket 199 2016-04-07T03:49:38.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24150 Milliseconds Vbucket 199 2016-04-07T03:49:38.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24180 Milliseconds Vbucket 199 2016-04-07T03:49:38.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24210 Milliseconds Vbucket 199 2016-04-07T03:49:38.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24240 Milliseconds Vbucket 199 2016-04-07T03:49:38.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24270 Milliseconds Vbucket 199 2016-04-07T03:49:38.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24300 Milliseconds Vbucket 199 2016-04-07T03:49:38.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24330 Milliseconds Vbucket 199 2016-04-07T03:49:38.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24360 Milliseconds Vbucket 199 2016-04-07T03:49:38.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24390 Milliseconds Vbucket 199 2016-04-07T03:49:38.769-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13860000 QueuedCount 49957 2016-04-07T03:49:38.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24420 Milliseconds Vbucket 199 2016-04-07T03:49:38.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24450 Milliseconds Vbucket 199 2016-04-07T03:49:38.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24480 Milliseconds Vbucket 199 2016-04-07T03:49:38.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24510 Milliseconds Vbucket 199 2016-04-07T03:49:38.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24540 Milliseconds Vbucket 199 2016-04-07T03:49:38.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24570 Milliseconds Vbucket 199 2016-04-07T03:49:38.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24600 Milliseconds Vbucket 199 2016-04-07T03:49:38.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24630 Milliseconds Vbucket 199 2016-04-07T03:49:39.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24660 Milliseconds Vbucket 199 2016-04-07T03:49:39.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24690 Milliseconds Vbucket 199 2016-04-07T03:49:39.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24720 Milliseconds Vbucket 199 2016-04-07T03:49:39.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24750 Milliseconds Vbucket 199 2016-04-07T03:49:39.133-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13870000 QueuedCount 39957 2016-04-07T03:49:39.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24780 Milliseconds Vbucket 199 2016-04-07T03:49:39.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24810 Milliseconds Vbucket 199 2016-04-07T03:49:39.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24840 Milliseconds Vbucket 199 2016-04-07T03:49:39.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24870 Milliseconds Vbucket 199 2016-04-07T03:49:39.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24900 Milliseconds Vbucket 199 2016-04-07T03:49:39.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24930 Milliseconds Vbucket 199 2016-04-07T03:49:39.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24960 Milliseconds Vbucket 199 2016-04-07T03:49:39.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 24990 Milliseconds Vbucket 199 2016-04-07T03:49:39.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25020 Milliseconds Vbucket 199 2016-04-07T03:49:39.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25050 Milliseconds Vbucket 199 2016-04-07T03:49:39.436-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13880000 QueuedCount 29957 2016-04-07T03:49:39.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25080 Milliseconds Vbucket 199 2016-04-07T03:49:39.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25110 Milliseconds Vbucket 199 2016-04-07T03:49:39.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25140 Milliseconds Vbucket 199 2016-04-07T03:49:39.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25170 Milliseconds Vbucket 199 2016-04-07T03:49:39.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25200 Milliseconds Vbucket 199 2016-04-07T03:49:39.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25230 Milliseconds Vbucket 199 2016-04-07T03:49:39.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25260 Milliseconds Vbucket 199 2016-04-07T03:49:39.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25290 Milliseconds Vbucket 199 2016-04-07T03:49:39.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25320 Milliseconds Vbucket 199 2016-04-07T03:49:39.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25350 Milliseconds Vbucket 199 2016-04-07T03:49:39.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25380 Milliseconds Vbucket 199 2016-04-07T03:49:39.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25410 Milliseconds Vbucket 199 2016-04-07T03:49:39.794-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13890000 QueuedCount 19957 2016-04-07T03:49:39.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25440 Milliseconds Vbucket 199 2016-04-07T03:49:39.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25470 Milliseconds Vbucket 199 2016-04-07T03:49:39.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25500 Milliseconds Vbucket 199 2016-04-07T03:49:39.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25530 Milliseconds Vbucket 199 2016-04-07T03:49:39.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25560 Milliseconds Vbucket 199 2016-04-07T03:49:39.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25590 Milliseconds Vbucket 199 2016-04-07T03:49:39.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25620 Milliseconds Vbucket 199 2016-04-07T03:49:40.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25650 Milliseconds Vbucket 199 2016-04-07T03:49:40.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25680 Milliseconds Vbucket 199 2016-04-07T03:49:40.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25710 Milliseconds Vbucket 199 2016-04-07T03:49:40.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25740 Milliseconds Vbucket 199 2016-04-07T03:49:40.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25770 Milliseconds Vbucket 199 2016-04-07T03:49:40.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25800 Milliseconds Vbucket 199 2016-04-07T03:49:40.165-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13900000 QueuedCount 9957 2016-04-07T03:49:40.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25830 Milliseconds Vbucket 199 2016-04-07T03:49:40.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25860 Milliseconds Vbucket 199 2016-04-07T03:49:40.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25890 Milliseconds Vbucket 199 2016-04-07T03:49:40.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25920 Milliseconds Vbucket 199 2016-04-07T03:49:40.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25950 Milliseconds Vbucket 199 2016-04-07T03:49:40.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 25980 Milliseconds Vbucket 199 2016-04-07T03:49:40.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26010 Milliseconds Vbucket 199 2016-04-07T03:49:40.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26040 Milliseconds Vbucket 199 2016-04-07T03:49:40.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26070 Milliseconds Vbucket 199 2016-04-07T03:49:40.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26100 Milliseconds Vbucket 199 2016-04-07T03:49:40.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26130 Milliseconds Vbucket 199 2016-04-07T03:49:40.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26160 Milliseconds Vbucket 199 2016-04-07T03:49:40.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26190 Milliseconds Vbucket 199 2016-04-07T03:49:40.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26220 Milliseconds Vbucket 199 2016-04-07T03:49:40.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26250 Milliseconds Vbucket 199 2016-04-07T03:49:40.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26280 Milliseconds Vbucket 199 2016-04-07T03:49:40.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26310 Milliseconds Vbucket 199 2016-04-07T03:49:40.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26340 Milliseconds Vbucket 199 2016-04-07T03:49:40.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26370 Milliseconds Vbucket 199 2016-04-07T03:49:40.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26400 Milliseconds Vbucket 199 2016-04-07T03:49:40.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26430 Milliseconds Vbucket 199 2016-04-07T03:49:40.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26460 Milliseconds Vbucket 199 2016-04-07T03:49:40.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26490 Milliseconds Vbucket 199 2016-04-07T03:49:40.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26520 Milliseconds Vbucket 199 2016-04-07T03:49:40.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26550 Milliseconds Vbucket 199 2016-04-07T03:49:40.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26580 Milliseconds Vbucket 199 2016-04-07T03:49:40.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26610 Milliseconds Vbucket 199 2016-04-07T03:49:41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26640 Milliseconds Vbucket 199 2016-04-07T03:49:41.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26670 Milliseconds Vbucket 199 2016-04-07T03:49:41.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26700 Milliseconds Vbucket 199 2016-04-07T03:49:41.061-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.371µs CommitTime 521.932098ms TotalFlushTime 9m10.220348344s TotalCommitTime 36.432662811s 2016-04-07T03:49:41.061-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 13909957, 13909957, 102 committed:true 2016-04-07T03:49:41.061-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 13909957, 13909957, 102 committed:true) SnapCreateDur 532.151118ms SnapOpenDur 56.548µs 2016-04-07T03:49:41.083-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13910000 QueuedCount 1485 2016-04-07T03:49:41.086-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26730 Milliseconds Vbucket 199 2016-04-07T03:49:41.111-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26760 Milliseconds Vbucket 199 2016-04-07T03:49:41.134-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26790 Milliseconds Vbucket 199 2016-04-07T03:49:41.158-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26820 Milliseconds Vbucket 199 2016-04-07T03:49:41.182-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.206-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26850 Milliseconds Vbucket 199 2016-04-07T03:49:41.229-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26880 Milliseconds Vbucket 199 2016-04-07T03:49:41.253-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26910 Milliseconds Vbucket 199 2016-04-07T03:49:41.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26940 Milliseconds Vbucket 199 2016-04-07T03:49:41.305-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 26970 Milliseconds Vbucket 199 2016-04-07T03:49:41.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27000 Milliseconds Vbucket 199 2016-04-07T03:49:41.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27030 Milliseconds Vbucket 199 2016-04-07T03:49:41.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27060 Milliseconds Vbucket 199 2016-04-07T03:49:41.429-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27090 Milliseconds Vbucket 199 2016-04-07T03:49:41.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27120 Milliseconds Vbucket 199 2016-04-07T03:49:41.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27150 Milliseconds Vbucket 199 2016-04-07T03:49:41.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27180 Milliseconds Vbucket 199 2016-04-07T03:49:41.548-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.55-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13920000 QueuedCount 49844 2016-04-07T03:49:41.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27210 Milliseconds Vbucket 199 2016-04-07T03:49:41.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27240 Milliseconds Vbucket 199 2016-04-07T03:49:41.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27270 Milliseconds Vbucket 199 2016-04-07T03:49:41.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27300 Milliseconds Vbucket 199 2016-04-07T03:49:41.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27330 Milliseconds Vbucket 199 2016-04-07T03:49:41.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27360 Milliseconds Vbucket 199 2016-04-07T03:49:41.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27390 Milliseconds Vbucket 199 2016-04-07T03:49:41.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27420 Milliseconds Vbucket 199 2016-04-07T03:49:41.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27450 Milliseconds Vbucket 199 2016-04-07T03:49:41.818-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:41.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27480 Milliseconds Vbucket 199 2016-04-07T03:49:41.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27510 Milliseconds Vbucket 199 2016-04-07T03:49:41.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27540 Milliseconds Vbucket 199 2016-04-07T03:49:41.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27570 Milliseconds Vbucket 199 2016-04-07T03:49:41.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27600 Milliseconds Vbucket 199 2016-04-07T03:49:41.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27630 Milliseconds Vbucket 199 2016-04-07T03:49:41.996-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:42.01-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13930000 QueuedCount 48761 2016-04-07T03:49:42.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27660 Milliseconds Vbucket 199 2016-04-07T03:49:42.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27690 Milliseconds Vbucket 199 2016-04-07T03:49:42.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27720 Milliseconds Vbucket 199 2016-04-07T03:49:42.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27750 Milliseconds Vbucket 199 2016-04-07T03:49:42.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27780 Milliseconds Vbucket 199 2016-04-07T03:49:42.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27810 Milliseconds Vbucket 199 2016-04-07T03:49:42.184-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:42.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27840 Milliseconds Vbucket 199 2016-04-07T03:49:42.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27870 Milliseconds Vbucket 199 2016-04-07T03:49:42.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27900 Milliseconds Vbucket 199 2016-04-07T03:49:42.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27930 Milliseconds Vbucket 199 2016-04-07T03:49:42.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27960 Milliseconds Vbucket 199 2016-04-07T03:49:42.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 27990 Milliseconds Vbucket 199 2016-04-07T03:49:42.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28020 Milliseconds Vbucket 199 2016-04-07T03:49:42.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28050 Milliseconds Vbucket 199 2016-04-07T03:49:42.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28080 Milliseconds Vbucket 199 2016-04-07T03:49:42.445-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13940000 QueuedCount 50000 2016-04-07T03:49:42.451-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:42.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28110 Milliseconds Vbucket 199 2016-04-07T03:49:42.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28140 Milliseconds Vbucket 199 2016-04-07T03:49:42.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28170 Milliseconds Vbucket 199 2016-04-07T03:49:42.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28200 Milliseconds Vbucket 199 2016-04-07T03:49:42.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28230 Milliseconds Vbucket 199 2016-04-07T03:49:42.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28260 Milliseconds Vbucket 199 2016-04-07T03:49:42.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28290 Milliseconds Vbucket 199 2016-04-07T03:49:42.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28320 Milliseconds Vbucket 199 2016-04-07T03:49:42.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28350 Milliseconds Vbucket 199 2016-04-07T03:49:42.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28380 Milliseconds Vbucket 199 2016-04-07T03:49:42.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28410 Milliseconds Vbucket 199 2016-04-07T03:49:42.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28440 Milliseconds Vbucket 199 2016-04-07T03:49:42.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28470 Milliseconds Vbucket 199 2016-04-07T03:49:42.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28500 Milliseconds Vbucket 199 2016-04-07T03:49:42.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28530 Milliseconds Vbucket 199 2016-04-07T03:49:42.894-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13950000 QueuedCount 50000 2016-04-07T03:49:42.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28560 Milliseconds Vbucket 199 2016-04-07T03:49:42.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28590 Milliseconds Vbucket 199 2016-04-07T03:49:42.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28620 Milliseconds Vbucket 199 2016-04-07T03:49:43.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28650 Milliseconds Vbucket 199 2016-04-07T03:49:43.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28680 Milliseconds Vbucket 199 2016-04-07T03:49:43.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28710 Milliseconds Vbucket 199 2016-04-07T03:49:43.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28740 Milliseconds Vbucket 199 2016-04-07T03:49:43.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28770 Milliseconds Vbucket 199 2016-04-07T03:49:43.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28800 Milliseconds Vbucket 199 2016-04-07T03:49:43.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28830 Milliseconds Vbucket 199 2016-04-07T03:49:43.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28860 Milliseconds Vbucket 199 2016-04-07T03:49:43.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28890 Milliseconds Vbucket 199 2016-04-07T03:49:43.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28920 Milliseconds Vbucket 199 2016-04-07T03:49:43.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28950 Milliseconds Vbucket 199 2016-04-07T03:49:43.318-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13960000 QueuedCount 50000 2016-04-07T03:49:43.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 28980 Milliseconds Vbucket 199 2016-04-07T03:49:43.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29010 Milliseconds Vbucket 199 2016-04-07T03:49:43.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29040 Milliseconds Vbucket 199 2016-04-07T03:49:43.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29070 Milliseconds Vbucket 199 2016-04-07T03:49:43.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29100 Milliseconds Vbucket 199 2016-04-07T03:49:43.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29130 Milliseconds Vbucket 199 2016-04-07T03:49:43.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29160 Milliseconds Vbucket 199 2016-04-07T03:49:43.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29190 Milliseconds Vbucket 199 2016-04-07T03:49:43.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29220 Milliseconds Vbucket 199 2016-04-07T03:49:43.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29250 Milliseconds Vbucket 199 2016-04-07T03:49:43.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29280 Milliseconds Vbucket 199 2016-04-07T03:49:43.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29310 Milliseconds Vbucket 199 2016-04-07T03:49:43.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29340 Milliseconds Vbucket 199 2016-04-07T03:49:43.719-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13970000 QueuedCount 50000 2016-04-07T03:49:43.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29370 Milliseconds Vbucket 199 2016-04-07T03:49:43.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29400 Milliseconds Vbucket 199 2016-04-07T03:49:43.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29430 Milliseconds Vbucket 199 2016-04-07T03:49:43.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29460 Milliseconds Vbucket 199 2016-04-07T03:49:43.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29490 Milliseconds Vbucket 199 2016-04-07T03:49:43.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29520 Milliseconds Vbucket 199 2016-04-07T03:49:43.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29550 Milliseconds Vbucket 199 2016-04-07T03:49:43.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29580 Milliseconds Vbucket 199 2016-04-07T03:49:43.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29610 Milliseconds Vbucket 199 2016-04-07T03:49:44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29640 Milliseconds Vbucket 199 2016-04-07T03:49:44-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:44.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29670 Milliseconds Vbucket 199 2016-04-07T03:49:44.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29700 Milliseconds Vbucket 199 2016-04-07T03:49:44.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29730 Milliseconds Vbucket 199 2016-04-07T03:49:44.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29760 Milliseconds Vbucket 199 2016-04-07T03:49:44.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29790 Milliseconds Vbucket 199 2016-04-07T03:49:44.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29820 Milliseconds Vbucket 199 2016-04-07T03:49:44.205-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13980000 QueuedCount 50000 2016-04-07T03:49:44.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29850 Milliseconds Vbucket 199 2016-04-07T03:49:44.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29880 Milliseconds Vbucket 199 2016-04-07T03:49:44.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29910 Milliseconds Vbucket 199 2016-04-07T03:49:44.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29940 Milliseconds Vbucket 199 2016-04-07T03:49:44.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 29970 Milliseconds Vbucket 199 2016-04-07T03:49:44.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30000 Milliseconds Vbucket 199 2016-04-07T03:49:44.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30030 Milliseconds Vbucket 199 2016-04-07T03:49:44.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30060 Milliseconds Vbucket 199 2016-04-07T03:49:44.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30090 Milliseconds Vbucket 199 2016-04-07T03:49:44.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30120 Milliseconds Vbucket 199 2016-04-07T03:49:44.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30150 Milliseconds Vbucket 199 2016-04-07T03:49:44.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30180 Milliseconds Vbucket 199 2016-04-07T03:49:44.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30210 Milliseconds Vbucket 199 2016-04-07T03:49:44.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30240 Milliseconds Vbucket 199 2016-04-07T03:49:44.62-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 13990000 QueuedCount 49304 2016-04-07T03:49:44.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30270 Milliseconds Vbucket 199 2016-04-07T03:49:44.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30300 Milliseconds Vbucket 199 2016-04-07T03:49:44.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30330 Milliseconds Vbucket 199 2016-04-07T03:49:44.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30360 Milliseconds Vbucket 199 2016-04-07T03:49:44.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30390 Milliseconds Vbucket 199 2016-04-07T03:49:44.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30420 Milliseconds Vbucket 199 2016-04-07T03:49:44.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30450 Milliseconds Vbucket 199 2016-04-07T03:49:44.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30480 Milliseconds Vbucket 199 2016-04-07T03:49:44.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30510 Milliseconds Vbucket 199 2016-04-07T03:49:44.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30540 Milliseconds Vbucket 199 2016-04-07T03:49:44.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30570 Milliseconds Vbucket 199 2016-04-07T03:49:44.945-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14000000 QueuedCount 39304 2016-04-07T03:49:44.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30600 Milliseconds Vbucket 199 2016-04-07T03:49:44.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30630 Milliseconds Vbucket 199 2016-04-07T03:49:45.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30660 Milliseconds Vbucket 199 2016-04-07T03:49:45.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30690 Milliseconds Vbucket 199 2016-04-07T03:49:45.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30720 Milliseconds Vbucket 199 2016-04-07T03:49:45.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30750 Milliseconds Vbucket 199 2016-04-07T03:49:45.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30780 Milliseconds Vbucket 199 2016-04-07T03:49:45.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30810 Milliseconds Vbucket 199 2016-04-07T03:49:45.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30840 Milliseconds Vbucket 199 2016-04-07T03:49:45.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30870 Milliseconds Vbucket 199 2016-04-07T03:49:45.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30900 Milliseconds Vbucket 199 2016-04-07T03:49:45.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30930 Milliseconds Vbucket 199 2016-04-07T03:49:45.315-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14010000 QueuedCount 29304 2016-04-07T03:49:45.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30960 Milliseconds Vbucket 199 2016-04-07T03:49:45.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 30990 Milliseconds Vbucket 199 2016-04-07T03:49:45.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31020 Milliseconds Vbucket 199 2016-04-07T03:49:45.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31050 Milliseconds Vbucket 199 2016-04-07T03:49:45.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31080 Milliseconds Vbucket 199 2016-04-07T03:49:45.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31110 Milliseconds Vbucket 199 2016-04-07T03:49:45.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31140 Milliseconds Vbucket 199 2016-04-07T03:49:45.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31170 Milliseconds Vbucket 199 2016-04-07T03:49:45.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31200 Milliseconds Vbucket 199 2016-04-07T03:49:45.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31230 Milliseconds Vbucket 199 2016-04-07T03:49:45.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31260 Milliseconds Vbucket 199 2016-04-07T03:49:45.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31290 Milliseconds Vbucket 199 2016-04-07T03:49:45.679-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14020000 QueuedCount 19304 2016-04-07T03:49:45.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31320 Milliseconds Vbucket 199 2016-04-07T03:49:45.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31350 Milliseconds Vbucket 199 2016-04-07T03:49:45.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31380 Milliseconds Vbucket 199 2016-04-07T03:49:45.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31410 Milliseconds Vbucket 199 2016-04-07T03:49:45.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31440 Milliseconds Vbucket 199 2016-04-07T03:49:45.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31470 Milliseconds Vbucket 199 2016-04-07T03:49:45.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31500 Milliseconds Vbucket 199 2016-04-07T03:49:45.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31530 Milliseconds Vbucket 199 2016-04-07T03:49:45.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31560 Milliseconds Vbucket 199 2016-04-07T03:49:45.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31590 Milliseconds Vbucket 199 2016-04-07T03:49:45.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31620 Milliseconds Vbucket 199 2016-04-07T03:49:46.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31650 Milliseconds Vbucket 199 2016-04-07T03:49:46.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31680 Milliseconds Vbucket 199 2016-04-07T03:49:46.061-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14030000 QueuedCount 9304 2016-04-07T03:49:46.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31710 Milliseconds Vbucket 199 2016-04-07T03:49:46.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31740 Milliseconds Vbucket 199 2016-04-07T03:49:46.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31770 Milliseconds Vbucket 199 2016-04-07T03:49:46.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31800 Milliseconds Vbucket 199 2016-04-07T03:49:46.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31830 Milliseconds Vbucket 199 2016-04-07T03:49:46.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31860 Milliseconds Vbucket 199 2016-04-07T03:49:46.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31890 Milliseconds Vbucket 199 2016-04-07T03:49:46.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31920 Milliseconds Vbucket 199 2016-04-07T03:49:46.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31950 Milliseconds Vbucket 199 2016-04-07T03:49:46.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 31980 Milliseconds Vbucket 199 2016-04-07T03:49:46.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32010 Milliseconds Vbucket 199 2016-04-07T03:49:46.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32040 Milliseconds Vbucket 199 2016-04-07T03:49:46.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32070 Milliseconds Vbucket 199 2016-04-07T03:49:46.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32100 Milliseconds Vbucket 199 2016-04-07T03:49:46.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32130 Milliseconds Vbucket 199 2016-04-07T03:49:46.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32160 Milliseconds Vbucket 199 2016-04-07T03:49:46.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32190 Milliseconds Vbucket 199 2016-04-07T03:49:46.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32220 Milliseconds Vbucket 199 2016-04-07T03:49:46.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32250 Milliseconds Vbucket 199 2016-04-07T03:49:46.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32280 Milliseconds Vbucket 199 2016-04-07T03:49:46.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32310 Milliseconds Vbucket 199 2016-04-07T03:49:46.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32340 Milliseconds Vbucket 199 2016-04-07T03:49:46.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32370 Milliseconds Vbucket 199 2016-04-07T03:49:46.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32400 Milliseconds Vbucket 199 2016-04-07T03:49:46.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32430 Milliseconds Vbucket 199 2016-04-07T03:49:46.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32460 Milliseconds Vbucket 199 2016-04-07T03:49:46.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32490 Milliseconds Vbucket 199 2016-04-07T03:49:46.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32520 Milliseconds Vbucket 199 2016-04-07T03:49:46.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32550 Milliseconds Vbucket 199 2016-04-07T03:49:46.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32580 Milliseconds Vbucket 199 2016-04-07T03:49:46.954-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.121µs CommitTime 511.83297ms TotalFlushTime 9m15.396657028s TotalCommitTime 36.944495781s 2016-04-07T03:49:46.954-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 14039304, 14039304, 103 committed:true 2016-04-07T03:49:46.954-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 14039304, 14039304, 103 committed:true) SnapCreateDur 522.938053ms SnapOpenDur 32.697µs 2016-04-07T03:49:46.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32610 Milliseconds Vbucket 199 2016-04-07T03:49:46.976-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:46.999-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32640 Milliseconds Vbucket 199 2016-04-07T03:49:47.022-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:47.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32670 Milliseconds Vbucket 199 2016-04-07T03:49:47.045-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:47.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32700 Milliseconds Vbucket 199 2016-04-07T03:49:47.067-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:47.089-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:47.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32730 Milliseconds Vbucket 199 2016-04-07T03:49:47.112-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:47.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32760 Milliseconds Vbucket 199 2016-04-07T03:49:47.136-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:47.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32790 Milliseconds Vbucket 199 2016-04-07T03:49:47.162-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:47.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32820 Milliseconds Vbucket 199 2016-04-07T03:49:47.184-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:47.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32850 Milliseconds Vbucket 199 2016-04-07T03:49:47.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32880 Milliseconds Vbucket 199 2016-04-07T03:49:47.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32910 Milliseconds Vbucket 199 2016-04-07T03:49:47.295-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:47.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32940 Milliseconds Vbucket 199 2016-04-07T03:49:47.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 32970 Milliseconds Vbucket 199 2016-04-07T03:49:47.336-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14040000 QueuedCount 50000 2016-04-07T03:49:47.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33000 Milliseconds Vbucket 199 2016-04-07T03:49:47.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33030 Milliseconds Vbucket 199 2016-04-07T03:49:47.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33060 Milliseconds Vbucket 199 2016-04-07T03:49:47.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33090 Milliseconds Vbucket 199 2016-04-07T03:49:47.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33120 Milliseconds Vbucket 199 2016-04-07T03:49:47.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33150 Milliseconds Vbucket 199 2016-04-07T03:49:47.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33180 Milliseconds Vbucket 199 2016-04-07T03:49:47.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33210 Milliseconds Vbucket 199 2016-04-07T03:49:47.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33240 Milliseconds Vbucket 199 2016-04-07T03:49:47.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33270 Milliseconds Vbucket 199 2016-04-07T03:49:47.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33300 Milliseconds Vbucket 199 2016-04-07T03:49:47.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33330 Milliseconds Vbucket 199 2016-04-07T03:49:47.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33360 Milliseconds Vbucket 199 2016-04-07T03:49:47.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33390 Milliseconds Vbucket 199 2016-04-07T03:49:47.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33420 Milliseconds Vbucket 199 2016-04-07T03:49:47.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33450 Milliseconds Vbucket 199 2016-04-07T03:49:47.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33480 Milliseconds Vbucket 199 2016-04-07T03:49:47.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33510 Milliseconds Vbucket 199 2016-04-07T03:49:47.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33540 Milliseconds Vbucket 199 2016-04-07T03:49:47.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33570 Milliseconds Vbucket 199 2016-04-07T03:49:47.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33600 Milliseconds Vbucket 199 2016-04-07T03:49:47.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33630 Milliseconds Vbucket 199 2016-04-07T03:49:48.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33660 Milliseconds Vbucket 199 2016-04-07T03:49:48.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33690 Milliseconds Vbucket 199 2016-04-07T03:49:48.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33720 Milliseconds Vbucket 199 2016-04-07T03:49:48.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33750 Milliseconds Vbucket 199 2016-04-07T03:49:48.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33780 Milliseconds Vbucket 199 2016-04-07T03:49:48.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33810 Milliseconds Vbucket 199 2016-04-07T03:49:48.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33840 Milliseconds Vbucket 199 2016-04-07T03:49:48.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33870 Milliseconds Vbucket 199 2016-04-07T03:49:48.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33900 Milliseconds Vbucket 199 2016-04-07T03:49:48.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33930 Milliseconds Vbucket 199 2016-04-07T03:49:48.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33960 Milliseconds Vbucket 199 2016-04-07T03:49:48.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 33990 Milliseconds Vbucket 199 2016-04-07T03:49:48.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34020 Milliseconds Vbucket 199 2016-04-07T03:49:48.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34050 Milliseconds Vbucket 199 2016-04-07T03:49:48.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34080 Milliseconds Vbucket 199 2016-04-07T03:49:48.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34110 Milliseconds Vbucket 199 2016-04-07T03:49:48.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34140 Milliseconds Vbucket 199 2016-04-07T03:49:48.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34170 Milliseconds Vbucket 199 2016-04-07T03:49:48.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34200 Milliseconds Vbucket 199 2016-04-07T03:49:48.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34230 Milliseconds Vbucket 199 2016-04-07T03:49:48.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34260 Milliseconds Vbucket 199 2016-04-07T03:49:48.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34290 Milliseconds Vbucket 199 2016-04-07T03:49:48.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34320 Milliseconds Vbucket 199 2016-04-07T03:49:48.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34350 Milliseconds Vbucket 199 2016-04-07T03:49:48.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34380 Milliseconds Vbucket 199 2016-04-07T03:49:48.768-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:48.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34410 Milliseconds Vbucket 199 2016-04-07T03:49:48.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34440 Milliseconds Vbucket 199 2016-04-07T03:49:48.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34470 Milliseconds Vbucket 199 2016-04-07T03:49:48.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34500 Milliseconds Vbucket 199 2016-04-07T03:49:48.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34530 Milliseconds Vbucket 199 2016-04-07T03:49:48.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34560 Milliseconds Vbucket 199 2016-04-07T03:49:48.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34590 Milliseconds Vbucket 199 2016-04-07T03:49:48.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34620 Milliseconds Vbucket 199 2016-04-07T03:49:49.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34650 Milliseconds Vbucket 199 2016-04-07T03:49:49.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34680 Milliseconds Vbucket 199 2016-04-07T03:49:49.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34710 Milliseconds Vbucket 199 2016-04-07T03:49:49.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34740 Milliseconds Vbucket 199 2016-04-07T03:49:49.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34770 Milliseconds Vbucket 199 2016-04-07T03:49:49.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34800 Milliseconds Vbucket 199 2016-04-07T03:49:49.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34830 Milliseconds Vbucket 199 2016-04-07T03:49:49.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34860 Milliseconds Vbucket 199 2016-04-07T03:49:49.22-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:49.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34890 Milliseconds Vbucket 199 2016-04-07T03:49:49.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34920 Milliseconds Vbucket 199 2016-04-07T03:49:49.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34950 Milliseconds Vbucket 199 2016-04-07T03:49:49.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 34980 Milliseconds Vbucket 199 2016-04-07T03:49:49.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35010 Milliseconds Vbucket 199 2016-04-07T03:49:49.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35040 Milliseconds Vbucket 199 2016-04-07T03:49:49.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35070 Milliseconds Vbucket 199 2016-04-07T03:49:49.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35100 Milliseconds Vbucket 199 2016-04-07T03:49:49.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35130 Milliseconds Vbucket 199 2016-04-07T03:49:49.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35160 Milliseconds Vbucket 199 2016-04-07T03:49:49.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35190 Milliseconds Vbucket 199 2016-04-07T03:49:49.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35220 Milliseconds Vbucket 199 2016-04-07T03:49:49.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35250 Milliseconds Vbucket 199 2016-04-07T03:49:49.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35280 Milliseconds Vbucket 199 2016-04-07T03:49:49.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35310 Milliseconds Vbucket 199 2016-04-07T03:49:49.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35340 Milliseconds Vbucket 199 2016-04-07T03:49:49.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35370 Milliseconds Vbucket 199 2016-04-07T03:49:49.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35400 Milliseconds Vbucket 199 2016-04-07T03:49:49.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35430 Milliseconds Vbucket 199 2016-04-07T03:49:49.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35460 Milliseconds Vbucket 199 2016-04-07T03:49:49.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35490 Milliseconds Vbucket 199 2016-04-07T03:49:49.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35520 Milliseconds Vbucket 199 2016-04-07T03:49:49.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35550 Milliseconds Vbucket 199 2016-04-07T03:49:49.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35580 Milliseconds Vbucket 199 2016-04-07T03:49:49.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35610 Milliseconds Vbucket 199 2016-04-07T03:49:50-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35640 Milliseconds Vbucket 199 2016-04-07T03:49:50.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35670 Milliseconds Vbucket 199 2016-04-07T03:49:50.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35700 Milliseconds Vbucket 199 2016-04-07T03:49:50.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35730 Milliseconds Vbucket 199 2016-04-07T03:49:50.097-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14050000 QueuedCount 50000 2016-04-07T03:49:50.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35760 Milliseconds Vbucket 199 2016-04-07T03:49:50.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35790 Milliseconds Vbucket 199 2016-04-07T03:49:50.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35820 Milliseconds Vbucket 199 2016-04-07T03:49:50.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35850 Milliseconds Vbucket 199 2016-04-07T03:49:50.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35880 Milliseconds Vbucket 199 2016-04-07T03:49:50.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35910 Milliseconds Vbucket 199 2016-04-07T03:49:50.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35940 Milliseconds Vbucket 199 2016-04-07T03:49:50.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 35970 Milliseconds Vbucket 199 2016-04-07T03:49:50.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36000 Milliseconds Vbucket 199 2016-04-07T03:49:50.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36030 Milliseconds Vbucket 199 2016-04-07T03:49:50.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36060 Milliseconds Vbucket 199 2016-04-07T03:49:50.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36090 Milliseconds Vbucket 199 2016-04-07T03:49:50.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36120 Milliseconds Vbucket 199 2016-04-07T03:49:50.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36150 Milliseconds Vbucket 199 2016-04-07T03:49:50.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36180 Milliseconds Vbucket 199 2016-04-07T03:49:50.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36210 Milliseconds Vbucket 199 2016-04-07T03:49:50.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36240 Milliseconds Vbucket 199 2016-04-07T03:49:50.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36270 Milliseconds Vbucket 199 2016-04-07T03:49:50.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36300 Milliseconds Vbucket 199 2016-04-07T03:49:50.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36330 Milliseconds Vbucket 199 2016-04-07T03:49:50.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36360 Milliseconds Vbucket 199 2016-04-07T03:49:50.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36390 Milliseconds Vbucket 199 2016-04-07T03:49:50.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36420 Milliseconds Vbucket 199 2016-04-07T03:49:50.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36450 Milliseconds Vbucket 199 2016-04-07T03:49:50.828-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14060000 QueuedCount 43432 2016-04-07T03:49:50.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36480 Milliseconds Vbucket 199 2016-04-07T03:49:50.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36510 Milliseconds Vbucket 199 2016-04-07T03:49:50.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36540 Milliseconds Vbucket 199 2016-04-07T03:49:50.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36570 Milliseconds Vbucket 199 2016-04-07T03:49:50.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36600 Milliseconds Vbucket 199 2016-04-07T03:49:50.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36630 Milliseconds Vbucket 199 2016-04-07T03:49:51.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36660 Milliseconds Vbucket 199 2016-04-07T03:49:51.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36690 Milliseconds Vbucket 199 2016-04-07T03:49:51.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36720 Milliseconds Vbucket 199 2016-04-07T03:49:51.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36750 Milliseconds Vbucket 199 2016-04-07T03:49:51.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36780 Milliseconds Vbucket 199 2016-04-07T03:49:51.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36810 Milliseconds Vbucket 199 2016-04-07T03:49:51.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36840 Milliseconds Vbucket 199 2016-04-07T03:49:51.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36870 Milliseconds Vbucket 199 2016-04-07T03:49:51.241-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14070000 QueuedCount 33432 2016-04-07T03:49:51.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36900 Milliseconds Vbucket 199 2016-04-07T03:49:51.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36930 Milliseconds Vbucket 199 2016-04-07T03:49:51.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36960 Milliseconds Vbucket 199 2016-04-07T03:49:51.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 36990 Milliseconds Vbucket 199 2016-04-07T03:49:51.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37020 Milliseconds Vbucket 199 2016-04-07T03:49:51.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37050 Milliseconds Vbucket 199 2016-04-07T03:49:51.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37080 Milliseconds Vbucket 199 2016-04-07T03:49:51.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37110 Milliseconds Vbucket 199 2016-04-07T03:49:51.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37140 Milliseconds Vbucket 199 2016-04-07T03:49:51.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37170 Milliseconds Vbucket 199 2016-04-07T03:49:51.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37200 Milliseconds Vbucket 199 2016-04-07T03:49:51.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37230 Milliseconds Vbucket 199 2016-04-07T03:49:51.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37260 Milliseconds Vbucket 199 2016-04-07T03:49:51.646-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14080000 QueuedCount 23432 2016-04-07T03:49:51.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37290 Milliseconds Vbucket 199 2016-04-07T03:49:51.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37320 Milliseconds Vbucket 199 2016-04-07T03:49:51.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37350 Milliseconds Vbucket 199 2016-04-07T03:49:51.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37380 Milliseconds Vbucket 199 2016-04-07T03:49:51.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37410 Milliseconds Vbucket 199 2016-04-07T03:49:51.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37440 Milliseconds Vbucket 199 2016-04-07T03:49:51.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37470 Milliseconds Vbucket 199 2016-04-07T03:49:51.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37500 Milliseconds Vbucket 199 2016-04-07T03:49:51.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37530 Milliseconds Vbucket 199 2016-04-07T03:49:51.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37560 Milliseconds Vbucket 199 2016-04-07T03:49:51.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37590 Milliseconds Vbucket 199 2016-04-07T03:49:51.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37620 Milliseconds Vbucket 199 2016-04-07T03:49:52.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37650 Milliseconds Vbucket 199 2016-04-07T03:49:52.038-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14090000 QueuedCount 13432 2016-04-07T03:49:52.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37680 Milliseconds Vbucket 199 2016-04-07T03:49:52.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37710 Milliseconds Vbucket 199 2016-04-07T03:49:52.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37740 Milliseconds Vbucket 199 2016-04-07T03:49:52.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37770 Milliseconds Vbucket 199 2016-04-07T03:49:52.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37800 Milliseconds Vbucket 199 2016-04-07T03:49:52.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37830 Milliseconds Vbucket 199 2016-04-07T03:49:52.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37860 Milliseconds Vbucket 199 2016-04-07T03:49:52.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37890 Milliseconds Vbucket 199 2016-04-07T03:49:52.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37920 Milliseconds Vbucket 199 2016-04-07T03:49:52.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37950 Milliseconds Vbucket 199 2016-04-07T03:49:52.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 37980 Milliseconds Vbucket 199 2016-04-07T03:49:52.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38010 Milliseconds Vbucket 199 2016-04-07T03:49:52.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38040 Milliseconds Vbucket 199 2016-04-07T03:49:52.421-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14100000 QueuedCount 3432 2016-04-07T03:49:52.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38070 Milliseconds Vbucket 199 2016-04-07T03:49:52.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38100 Milliseconds Vbucket 199 2016-04-07T03:49:52.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38130 Milliseconds Vbucket 199 2016-04-07T03:49:52.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38160 Milliseconds Vbucket 199 2016-04-07T03:49:52.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38190 Milliseconds Vbucket 199 2016-04-07T03:49:52.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38220 Milliseconds Vbucket 199 2016-04-07T03:49:52.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38250 Milliseconds Vbucket 199 2016-04-07T03:49:52.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38280 Milliseconds Vbucket 199 2016-04-07T03:49:52.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38310 Milliseconds Vbucket 199 2016-04-07T03:49:52.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38340 Milliseconds Vbucket 199 2016-04-07T03:49:52.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38370 Milliseconds Vbucket 199 2016-04-07T03:49:52.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38400 Milliseconds Vbucket 199 2016-04-07T03:49:52.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38430 Milliseconds Vbucket 199 2016-04-07T03:49:52.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38460 Milliseconds Vbucket 199 2016-04-07T03:49:52.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38490 Milliseconds Vbucket 199 2016-04-07T03:49:52.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38520 Milliseconds Vbucket 199 2016-04-07T03:49:52.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38550 Milliseconds Vbucket 199 2016-04-07T03:49:52.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38580 Milliseconds Vbucket 199 2016-04-07T03:49:52.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38610 Milliseconds Vbucket 199 2016-04-07T03:49:52.981-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.554µs CommitTime 393.994498ms TotalFlushTime 9m20.941153442s TotalCommitTime 37.338490279s 2016-04-07T03:49:52.981-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 14103432, 14103432, 104 committed:true 2016-04-07T03:49:52.981-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 14103432, 14103432, 104 committed:true) SnapCreateDur 404.222022ms SnapOpenDur 49.06µs 2016-04-07T03:49:53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38640 Milliseconds Vbucket 199 2016-04-07T03:49:53.005-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:53.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38670 Milliseconds Vbucket 199 2016-04-07T03:49:53.031-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:53.059-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:53.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38700 Milliseconds Vbucket 199 2016-04-07T03:49:53.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38730 Milliseconds Vbucket 199 2016-04-07T03:49:53.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38760 Milliseconds Vbucket 199 2016-04-07T03:49:53.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38790 Milliseconds Vbucket 199 2016-04-07T03:49:53.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38820 Milliseconds Vbucket 199 2016-04-07T03:49:53.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38850 Milliseconds Vbucket 199 2016-04-07T03:49:53.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38880 Milliseconds Vbucket 199 2016-04-07T03:49:53.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38910 Milliseconds Vbucket 199 2016-04-07T03:49:53.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38940 Milliseconds Vbucket 199 2016-04-07T03:49:53.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 38970 Milliseconds Vbucket 199 2016-04-07T03:49:53.338-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14110000 QueuedCount 50000 2016-04-07T03:49:53.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39000 Milliseconds Vbucket 199 2016-04-07T03:49:53.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39030 Milliseconds Vbucket 199 2016-04-07T03:49:53.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39060 Milliseconds Vbucket 199 2016-04-07T03:49:53.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39090 Milliseconds Vbucket 199 2016-04-07T03:49:53.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39120 Milliseconds Vbucket 199 2016-04-07T03:49:53.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39150 Milliseconds Vbucket 199 2016-04-07T03:49:53.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39180 Milliseconds Vbucket 199 2016-04-07T03:49:53.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39210 Milliseconds Vbucket 199 2016-04-07T03:49:53.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39240 Milliseconds Vbucket 199 2016-04-07T03:49:53.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39270 Milliseconds Vbucket 199 2016-04-07T03:49:53.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39300 Milliseconds Vbucket 199 2016-04-07T03:49:53.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39330 Milliseconds Vbucket 199 2016-04-07T03:49:53.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39360 Milliseconds Vbucket 199 2016-04-07T03:49:53.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39390 Milliseconds Vbucket 199 2016-04-07T03:49:53.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39420 Milliseconds Vbucket 199 2016-04-07T03:49:53.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39450 Milliseconds Vbucket 199 2016-04-07T03:49:53.83-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14120000 QueuedCount 50000 2016-04-07T03:49:53.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39480 Milliseconds Vbucket 199 2016-04-07T03:49:53.854-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:53.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39510 Milliseconds Vbucket 199 2016-04-07T03:49:53.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39540 Milliseconds Vbucket 199 2016-04-07T03:49:53.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39570 Milliseconds Vbucket 199 2016-04-07T03:49:53.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39600 Milliseconds Vbucket 199 2016-04-07T03:49:53.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39630 Milliseconds Vbucket 199 2016-04-07T03:49:54.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39660 Milliseconds Vbucket 199 2016-04-07T03:49:54.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39690 Milliseconds Vbucket 199 2016-04-07T03:49:54.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39720 Milliseconds Vbucket 199 2016-04-07T03:49:54.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39750 Milliseconds Vbucket 199 2016-04-07T03:49:54.117-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:54.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39780 Milliseconds Vbucket 199 2016-04-07T03:49:54.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39810 Milliseconds Vbucket 199 2016-04-07T03:49:54.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39840 Milliseconds Vbucket 199 2016-04-07T03:49:54.225-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14130000 QueuedCount 50000 2016-04-07T03:49:54.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39870 Milliseconds Vbucket 199 2016-04-07T03:49:54.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39900 Milliseconds Vbucket 199 2016-04-07T03:49:54.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39930 Milliseconds Vbucket 199 2016-04-07T03:49:54.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39960 Milliseconds Vbucket 199 2016-04-07T03:49:54.328-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:54.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 39990 Milliseconds Vbucket 199 2016-04-07T03:49:54.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40020 Milliseconds Vbucket 199 2016-04-07T03:49:54.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40050 Milliseconds Vbucket 199 2016-04-07T03:49:54.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40080 Milliseconds Vbucket 199 2016-04-07T03:49:54.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40110 Milliseconds Vbucket 199 2016-04-07T03:49:54.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40140 Milliseconds Vbucket 199 2016-04-07T03:49:54.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40170 Milliseconds Vbucket 199 2016-04-07T03:49:54.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40200 Milliseconds Vbucket 199 2016-04-07T03:49:54.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40230 Milliseconds Vbucket 199 2016-04-07T03:49:54.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40260 Milliseconds Vbucket 199 2016-04-07T03:49:54.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40290 Milliseconds Vbucket 199 2016-04-07T03:49:54.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40320 Milliseconds Vbucket 199 2016-04-07T03:49:54.704-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14140000 QueuedCount 48737 2016-04-07T03:49:54.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40350 Milliseconds Vbucket 199 2016-04-07T03:49:54.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40380 Milliseconds Vbucket 199 2016-04-07T03:49:54.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40410 Milliseconds Vbucket 199 2016-04-07T03:49:54.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40440 Milliseconds Vbucket 199 2016-04-07T03:49:54.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40470 Milliseconds Vbucket 199 2016-04-07T03:49:54.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40500 Milliseconds Vbucket 199 2016-04-07T03:49:54.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40530 Milliseconds Vbucket 199 2016-04-07T03:49:54.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40560 Milliseconds Vbucket 199 2016-04-07T03:49:54.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40590 Milliseconds Vbucket 199 2016-04-07T03:49:54.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40620 Milliseconds Vbucket 199 2016-04-07T03:49:55.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40650 Milliseconds Vbucket 199 2016-04-07T03:49:55.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40680 Milliseconds Vbucket 199 2016-04-07T03:49:55.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40710 Milliseconds Vbucket 199 2016-04-07T03:49:55.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40740 Milliseconds Vbucket 199 2016-04-07T03:49:55.113-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14150000 QueuedCount 38737 2016-04-07T03:49:55.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40770 Milliseconds Vbucket 199 2016-04-07T03:49:55.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40800 Milliseconds Vbucket 199 2016-04-07T03:49:55.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40830 Milliseconds Vbucket 199 2016-04-07T03:49:55.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40860 Milliseconds Vbucket 199 2016-04-07T03:49:55.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40890 Milliseconds Vbucket 199 2016-04-07T03:49:55.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40920 Milliseconds Vbucket 199 2016-04-07T03:49:55.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40950 Milliseconds Vbucket 199 2016-04-07T03:49:55.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 40980 Milliseconds Vbucket 199 2016-04-07T03:49:55.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41010 Milliseconds Vbucket 199 2016-04-07T03:49:55.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41040 Milliseconds Vbucket 199 2016-04-07T03:49:55.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41070 Milliseconds Vbucket 199 2016-04-07T03:49:55.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41100 Milliseconds Vbucket 199 2016-04-07T03:49:55.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41130 Milliseconds Vbucket 199 2016-04-07T03:49:55.518-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14160000 QueuedCount 28737 2016-04-07T03:49:55.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41160 Milliseconds Vbucket 199 2016-04-07T03:49:55.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41190 Milliseconds Vbucket 199 2016-04-07T03:49:55.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41220 Milliseconds Vbucket 199 2016-04-07T03:49:55.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41250 Milliseconds Vbucket 199 2016-04-07T03:49:55.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41280 Milliseconds Vbucket 199 2016-04-07T03:49:55.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41310 Milliseconds Vbucket 199 2016-04-07T03:49:55.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41340 Milliseconds Vbucket 199 2016-04-07T03:49:55.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41370 Milliseconds Vbucket 199 2016-04-07T03:49:55.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41400 Milliseconds Vbucket 199 2016-04-07T03:49:55.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41430 Milliseconds Vbucket 199 2016-04-07T03:49:55.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41460 Milliseconds Vbucket 199 2016-04-07T03:49:55.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41490 Milliseconds Vbucket 199 2016-04-07T03:49:55.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41520 Milliseconds Vbucket 199 2016-04-07T03:49:55.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41550 Milliseconds Vbucket 199 2016-04-07T03:49:55.915-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14170000 QueuedCount 18737 2016-04-07T03:49:55.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41580 Milliseconds Vbucket 199 2016-04-07T03:49:55.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41610 Milliseconds Vbucket 199 2016-04-07T03:49:56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41640 Milliseconds Vbucket 199 2016-04-07T03:49:56.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41670 Milliseconds Vbucket 199 2016-04-07T03:49:56.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41700 Milliseconds Vbucket 199 2016-04-07T03:49:56.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41730 Milliseconds Vbucket 199 2016-04-07T03:49:56.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41760 Milliseconds Vbucket 199 2016-04-07T03:49:56.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41790 Milliseconds Vbucket 199 2016-04-07T03:49:56.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41820 Milliseconds Vbucket 199 2016-04-07T03:49:56.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41850 Milliseconds Vbucket 199 2016-04-07T03:49:56.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41880 Milliseconds Vbucket 199 2016-04-07T03:49:56.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41910 Milliseconds Vbucket 199 2016-04-07T03:49:56.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41940 Milliseconds Vbucket 199 2016-04-07T03:49:56.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 41970 Milliseconds Vbucket 199 2016-04-07T03:49:56.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42000 Milliseconds Vbucket 199 2016-04-07T03:49:56.377-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14180000 QueuedCount 8737 2016-04-07T03:49:56.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42030 Milliseconds Vbucket 199 2016-04-07T03:49:56.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42060 Milliseconds Vbucket 199 2016-04-07T03:49:56.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42090 Milliseconds Vbucket 199 2016-04-07T03:49:56.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42120 Milliseconds Vbucket 199 2016-04-07T03:49:56.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42150 Milliseconds Vbucket 199 2016-04-07T03:49:56.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42180 Milliseconds Vbucket 199 2016-04-07T03:49:56.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42210 Milliseconds Vbucket 199 2016-04-07T03:49:56.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42240 Milliseconds Vbucket 199 2016-04-07T03:49:56.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42270 Milliseconds Vbucket 199 2016-04-07T03:49:56.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42300 Milliseconds Vbucket 199 2016-04-07T03:49:56.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42330 Milliseconds Vbucket 199 2016-04-07T03:49:56.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42360 Milliseconds Vbucket 199 2016-04-07T03:49:56.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42390 Milliseconds Vbucket 199 2016-04-07T03:49:56.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42420 Milliseconds Vbucket 199 2016-04-07T03:49:56.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42450 Milliseconds Vbucket 199 2016-04-07T03:49:56.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42480 Milliseconds Vbucket 199 2016-04-07T03:49:56.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42510 Milliseconds Vbucket 199 2016-04-07T03:49:56.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42540 Milliseconds Vbucket 199 2016-04-07T03:49:56.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42570 Milliseconds Vbucket 199 2016-04-07T03:49:56.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42600 Milliseconds Vbucket 199 2016-04-07T03:49:56.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42630 Milliseconds Vbucket 199 2016-04-07T03:49:57.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42660 Milliseconds Vbucket 199 2016-04-07T03:49:57.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42690 Milliseconds Vbucket 199 2016-04-07T03:49:57.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42720 Milliseconds Vbucket 199 2016-04-07T03:49:57.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42750 Milliseconds Vbucket 199 2016-04-07T03:49:57.138-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.41µs CommitTime 438.758585ms TotalFlushTime 9m24.529408553s TotalCommitTime 37.777248864s 2016-04-07T03:49:57.138-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 14188737, 14188737, 105 committed:true 2016-04-07T03:49:57.138-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 14188737, 14188737, 105 committed:true) SnapCreateDur 448.735437ms SnapOpenDur 46.783µs 2016-04-07T03:49:57.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42780 Milliseconds Vbucket 199 2016-04-07T03:49:57.159-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:57.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42810 Milliseconds Vbucket 199 2016-04-07T03:49:57.182-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:57.193-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14190000 QueuedCount 5550 2016-04-07T03:49:57.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42840 Milliseconds Vbucket 199 2016-04-07T03:49:57.204-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:57.228-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:57.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42870 Milliseconds Vbucket 199 2016-04-07T03:49:57.252-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:57.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42900 Milliseconds Vbucket 199 2016-04-07T03:49:57.284-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:57.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42930 Milliseconds Vbucket 199 2016-04-07T03:49:57.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42960 Milliseconds Vbucket 199 2016-04-07T03:49:57.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 42990 Milliseconds Vbucket 199 2016-04-07T03:49:57.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43020 Milliseconds Vbucket 199 2016-04-07T03:49:57.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43050 Milliseconds Vbucket 199 2016-04-07T03:49:57.425-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:57.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43080 Milliseconds Vbucket 199 2016-04-07T03:49:57.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43110 Milliseconds Vbucket 199 2016-04-07T03:49:57.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43140 Milliseconds Vbucket 199 2016-04-07T03:49:57.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43170 Milliseconds Vbucket 199 2016-04-07T03:49:57.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43200 Milliseconds Vbucket 199 2016-04-07T03:49:57.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43230 Milliseconds Vbucket 199 2016-04-07T03:49:57.618-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:57.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43260 Milliseconds Vbucket 199 2016-04-07T03:49:57.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43290 Milliseconds Vbucket 199 2016-04-07T03:49:57.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43320 Milliseconds Vbucket 199 2016-04-07T03:49:57.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43350 Milliseconds Vbucket 199 2016-04-07T03:49:57.711-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14200000 QueuedCount 50000 2016-04-07T03:49:57.722-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:57.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43380 Milliseconds Vbucket 199 2016-04-07T03:49:57.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43410 Milliseconds Vbucket 199 2016-04-07T03:49:57.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43440 Milliseconds Vbucket 199 2016-04-07T03:49:57.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43470 Milliseconds Vbucket 199 2016-04-07T03:49:57.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43500 Milliseconds Vbucket 199 2016-04-07T03:49:57.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43530 Milliseconds Vbucket 199 2016-04-07T03:49:57.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43560 Milliseconds Vbucket 199 2016-04-07T03:49:57.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43590 Milliseconds Vbucket 199 2016-04-07T03:49:57.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43620 Milliseconds Vbucket 199 2016-04-07T03:49:57.996-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:58.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43650 Milliseconds Vbucket 199 2016-04-07T03:49:58.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43680 Milliseconds Vbucket 199 2016-04-07T03:49:58.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43710 Milliseconds Vbucket 199 2016-04-07T03:49:58.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43740 Milliseconds Vbucket 199 2016-04-07T03:49:58.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43770 Milliseconds Vbucket 199 2016-04-07T03:49:58.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43800 Milliseconds Vbucket 199 2016-04-07T03:49:58.185-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14210000 QueuedCount 50000 2016-04-07T03:49:58.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43830 Milliseconds Vbucket 199 2016-04-07T03:49:58.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43860 Milliseconds Vbucket 199 2016-04-07T03:49:58.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43890 Milliseconds Vbucket 199 2016-04-07T03:49:58.275-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:58.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43920 Milliseconds Vbucket 199 2016-04-07T03:49:58.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43950 Milliseconds Vbucket 199 2016-04-07T03:49:58.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 43980 Milliseconds Vbucket 199 2016-04-07T03:49:58.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44010 Milliseconds Vbucket 199 2016-04-07T03:49:58.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44040 Milliseconds Vbucket 199 2016-04-07T03:49:58.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44070 Milliseconds Vbucket 199 2016-04-07T03:49:58.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44100 Milliseconds Vbucket 199 2016-04-07T03:49:58.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44130 Milliseconds Vbucket 199 2016-04-07T03:49:58.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44160 Milliseconds Vbucket 199 2016-04-07T03:49:58.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44190 Milliseconds Vbucket 199 2016-04-07T03:49:58.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44220 Milliseconds Vbucket 199 2016-04-07T03:49:58.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44250 Milliseconds Vbucket 199 2016-04-07T03:49:58.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44280 Milliseconds Vbucket 199 2016-04-07T03:49:58.659-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:58.662-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14220000 QueuedCount 49748 2016-04-07T03:49:58.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44310 Milliseconds Vbucket 199 2016-04-07T03:49:58.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44340 Milliseconds Vbucket 199 2016-04-07T03:49:58.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44370 Milliseconds Vbucket 199 2016-04-07T03:49:58.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44400 Milliseconds Vbucket 199 2016-04-07T03:49:58.773-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:58.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44430 Milliseconds Vbucket 199 2016-04-07T03:49:58.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44460 Milliseconds Vbucket 199 2016-04-07T03:49:58.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44490 Milliseconds Vbucket 199 2016-04-07T03:49:58.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44520 Milliseconds Vbucket 199 2016-04-07T03:49:58.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44550 Milliseconds Vbucket 199 2016-04-07T03:49:58.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44580 Milliseconds Vbucket 199 2016-04-07T03:49:58.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44610 Milliseconds Vbucket 199 2016-04-07T03:49:59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44640 Milliseconds Vbucket 199 2016-04-07T03:49:59.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44670 Milliseconds Vbucket 199 2016-04-07T03:49:59.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44700 Milliseconds Vbucket 199 2016-04-07T03:49:59.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44730 Milliseconds Vbucket 199 2016-04-07T03:49:59.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44760 Milliseconds Vbucket 199 2016-04-07T03:49:59.12-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14230000 QueuedCount 50000 2016-04-07T03:49:59.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44790 Milliseconds Vbucket 199 2016-04-07T03:49:59.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44820 Milliseconds Vbucket 199 2016-04-07T03:49:59.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44850 Milliseconds Vbucket 199 2016-04-07T03:49:59.22-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:59.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44880 Milliseconds Vbucket 199 2016-04-07T03:49:59.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44910 Milliseconds Vbucket 199 2016-04-07T03:49:59.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44940 Milliseconds Vbucket 199 2016-04-07T03:49:59.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 44970 Milliseconds Vbucket 199 2016-04-07T03:49:59.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45000 Milliseconds Vbucket 199 2016-04-07T03:49:59.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45030 Milliseconds Vbucket 199 2016-04-07T03:49:59.399-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:49:59.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45060 Milliseconds Vbucket 199 2016-04-07T03:49:59.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45090 Milliseconds Vbucket 199 2016-04-07T03:49:59.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45120 Milliseconds Vbucket 199 2016-04-07T03:49:59.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45150 Milliseconds Vbucket 199 2016-04-07T03:49:59.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45180 Milliseconds Vbucket 199 2016-04-07T03:49:59.552-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14240000 QueuedCount 50000 2016-04-07T03:49:59.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45210 Milliseconds Vbucket 199 2016-04-07T03:49:59.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45240 Milliseconds Vbucket 199 2016-04-07T03:49:59.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45270 Milliseconds Vbucket 199 2016-04-07T03:49:59.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45300 Milliseconds Vbucket 199 2016-04-07T03:49:59.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45330 Milliseconds Vbucket 199 2016-04-07T03:49:59.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45360 Milliseconds Vbucket 199 2016-04-07T03:49:59.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45390 Milliseconds Vbucket 199 2016-04-07T03:49:59.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45420 Milliseconds Vbucket 199 2016-04-07T03:49:59.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45450 Milliseconds Vbucket 199 2016-04-07T03:49:59.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45480 Milliseconds Vbucket 199 2016-04-07T03:49:59.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45510 Milliseconds Vbucket 199 2016-04-07T03:49:59.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45540 Milliseconds Vbucket 199 2016-04-07T03:49:59.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45570 Milliseconds Vbucket 199 2016-04-07T03:49:59.945-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14250000 QueuedCount 50000 2016-04-07T03:49:59.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45600 Milliseconds Vbucket 199 2016-04-07T03:49:59.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45630 Milliseconds Vbucket 199 2016-04-07T03:50:00.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45660 Milliseconds Vbucket 199 2016-04-07T03:50:00.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45690 Milliseconds Vbucket 199 2016-04-07T03:50:00.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45720 Milliseconds Vbucket 199 2016-04-07T03:50:00.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45750 Milliseconds Vbucket 199 2016-04-07T03:50:00.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45780 Milliseconds Vbucket 199 2016-04-07T03:50:00.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45810 Milliseconds Vbucket 199 2016-04-07T03:50:00.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45840 Milliseconds Vbucket 199 2016-04-07T03:50:00.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45870 Milliseconds Vbucket 199 2016-04-07T03:50:00.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45900 Milliseconds Vbucket 199 2016-04-07T03:50:00.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45930 Milliseconds Vbucket 199 2016-04-07T03:50:00.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45960 Milliseconds Vbucket 199 2016-04-07T03:50:00.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 45990 Milliseconds Vbucket 199 2016-04-07T03:50:00.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46020 Milliseconds Vbucket 199 2016-04-07T03:50:00.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46050 Milliseconds Vbucket 199 2016-04-07T03:50:00.411-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14260000 QueuedCount 50000 2016-04-07T03:50:00.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46080 Milliseconds Vbucket 199 2016-04-07T03:50:00.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46110 Milliseconds Vbucket 199 2016-04-07T03:50:00.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46140 Milliseconds Vbucket 199 2016-04-07T03:50:00.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46170 Milliseconds Vbucket 199 2016-04-07T03:50:00.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46200 Milliseconds Vbucket 199 2016-04-07T03:50:00.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46230 Milliseconds Vbucket 199 2016-04-07T03:50:00.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46260 Milliseconds Vbucket 199 2016-04-07T03:50:00.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46290 Milliseconds Vbucket 199 2016-04-07T03:50:00.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46320 Milliseconds Vbucket 199 2016-04-07T03:50:00.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46350 Milliseconds Vbucket 199 2016-04-07T03:50:00.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46380 Milliseconds Vbucket 199 2016-04-07T03:50:00.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46410 Milliseconds Vbucket 199 2016-04-07T03:50:00.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46440 Milliseconds Vbucket 199 2016-04-07T03:50:00.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46470 Milliseconds Vbucket 199 2016-04-07T03:50:00.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46500 Milliseconds Vbucket 199 2016-04-07T03:50:00.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46530 Milliseconds Vbucket 199 2016-04-07T03:50:00.905-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14270000 QueuedCount 50000 2016-04-07T03:50:00.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46560 Milliseconds Vbucket 199 2016-04-07T03:50:00.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46590 Milliseconds Vbucket 199 2016-04-07T03:50:00.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46620 Milliseconds Vbucket 199 2016-04-07T03:50:01.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46650 Milliseconds Vbucket 199 2016-04-07T03:50:01.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46680 Milliseconds Vbucket 199 2016-04-07T03:50:01.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46710 Milliseconds Vbucket 199 2016-04-07T03:50:01.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46740 Milliseconds Vbucket 199 2016-04-07T03:50:01.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46770 Milliseconds Vbucket 199 2016-04-07T03:50:01.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46800 Milliseconds Vbucket 199 2016-04-07T03:50:01.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46830 Milliseconds Vbucket 199 2016-04-07T03:50:01.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46860 Milliseconds Vbucket 199 2016-04-07T03:50:01.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46890 Milliseconds Vbucket 199 2016-04-07T03:50:01.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46920 Milliseconds Vbucket 199 2016-04-07T03:50:01.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46950 Milliseconds Vbucket 199 2016-04-07T03:50:01.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 46980 Milliseconds Vbucket 199 2016-04-07T03:50:01.366-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14280000 QueuedCount 45971 2016-04-07T03:50:01.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47010 Milliseconds Vbucket 199 2016-04-07T03:50:01.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47040 Milliseconds Vbucket 199 2016-04-07T03:50:01.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47070 Milliseconds Vbucket 199 2016-04-07T03:50:01.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47100 Milliseconds Vbucket 199 2016-04-07T03:50:01.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47130 Milliseconds Vbucket 199 2016-04-07T03:50:01.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47160 Milliseconds Vbucket 199 2016-04-07T03:50:01.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47190 Milliseconds Vbucket 199 2016-04-07T03:50:01.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47220 Milliseconds Vbucket 199 2016-04-07T03:50:01.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47250 Milliseconds Vbucket 199 2016-04-07T03:50:01.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47280 Milliseconds Vbucket 199 2016-04-07T03:50:01.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47310 Milliseconds Vbucket 199 2016-04-07T03:50:01.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47340 Milliseconds Vbucket 199 2016-04-07T03:50:01.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47370 Milliseconds Vbucket 199 2016-04-07T03:50:01.752-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14290000 QueuedCount 35971 2016-04-07T03:50:01.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47400 Milliseconds Vbucket 199 2016-04-07T03:50:01.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47430 Milliseconds Vbucket 199 2016-04-07T03:50:01.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47460 Milliseconds Vbucket 199 2016-04-07T03:50:01.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47490 Milliseconds Vbucket 199 2016-04-07T03:50:01.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47520 Milliseconds Vbucket 199 2016-04-07T03:50:01.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47550 Milliseconds Vbucket 199 2016-04-07T03:50:01.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47580 Milliseconds Vbucket 199 2016-04-07T03:50:01.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47610 Milliseconds Vbucket 199 2016-04-07T03:50:02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47640 Milliseconds Vbucket 199 2016-04-07T03:50:02.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47670 Milliseconds Vbucket 199 2016-04-07T03:50:02.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47700 Milliseconds Vbucket 199 2016-04-07T03:50:02.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47730 Milliseconds Vbucket 199 2016-04-07T03:50:02.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47760 Milliseconds Vbucket 199 2016-04-07T03:50:02.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47790 Milliseconds Vbucket 199 2016-04-07T03:50:02.151-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14300000 QueuedCount 25971 2016-04-07T03:50:02.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47820 Milliseconds Vbucket 199 2016-04-07T03:50:02.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47850 Milliseconds Vbucket 199 2016-04-07T03:50:02.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47880 Milliseconds Vbucket 199 2016-04-07T03:50:02.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47910 Milliseconds Vbucket 199 2016-04-07T03:50:02.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47940 Milliseconds Vbucket 199 2016-04-07T03:50:02.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 47970 Milliseconds Vbucket 199 2016-04-07T03:50:02.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48000 Milliseconds Vbucket 199 2016-04-07T03:50:02.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48030 Milliseconds Vbucket 199 2016-04-07T03:50:02.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48060 Milliseconds Vbucket 199 2016-04-07T03:50:02.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48090 Milliseconds Vbucket 199 2016-04-07T03:50:02.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48120 Milliseconds Vbucket 199 2016-04-07T03:50:02.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48150 Milliseconds Vbucket 199 2016-04-07T03:50:02.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48180 Milliseconds Vbucket 199 2016-04-07T03:50:02.551-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14310000 QueuedCount 15971 2016-04-07T03:50:02.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48210 Milliseconds Vbucket 199 2016-04-07T03:50:02.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48240 Milliseconds Vbucket 199 2016-04-07T03:50:02.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48270 Milliseconds Vbucket 199 2016-04-07T03:50:02.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48300 Milliseconds Vbucket 199 2016-04-07T03:50:02.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48330 Milliseconds Vbucket 199 2016-04-07T03:50:02.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48360 Milliseconds Vbucket 199 2016-04-07T03:50:02.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48390 Milliseconds Vbucket 199 2016-04-07T03:50:02.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48420 Milliseconds Vbucket 199 2016-04-07T03:50:02.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48450 Milliseconds Vbucket 199 2016-04-07T03:50:02.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48480 Milliseconds Vbucket 199 2016-04-07T03:50:02.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48510 Milliseconds Vbucket 199 2016-04-07T03:50:02.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48540 Milliseconds Vbucket 199 2016-04-07T03:50:02.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48570 Milliseconds Vbucket 199 2016-04-07T03:50:02.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48600 Milliseconds Vbucket 199 2016-04-07T03:50:02.97-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14320000 QueuedCount 5971 2016-04-07T03:50:02.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48630 Milliseconds Vbucket 199 2016-04-07T03:50:03.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48660 Milliseconds Vbucket 199 2016-04-07T03:50:03.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48690 Milliseconds Vbucket 199 2016-04-07T03:50:03.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48720 Milliseconds Vbucket 199 2016-04-07T03:50:03.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48750 Milliseconds Vbucket 199 2016-04-07T03:50:03.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48780 Milliseconds Vbucket 199 2016-04-07T03:50:03.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48810 Milliseconds Vbucket 199 2016-04-07T03:50:03.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48840 Milliseconds Vbucket 199 2016-04-07T03:50:03.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48870 Milliseconds Vbucket 199 2016-04-07T03:50:03.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48900 Milliseconds Vbucket 199 2016-04-07T03:50:03.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48930 Milliseconds Vbucket 199 2016-04-07T03:50:03.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48960 Milliseconds Vbucket 199 2016-04-07T03:50:03.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 48990 Milliseconds Vbucket 199 2016-04-07T03:50:03.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49020 Milliseconds Vbucket 199 2016-04-07T03:50:03.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49050 Milliseconds Vbucket 199 2016-04-07T03:50:03.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49080 Milliseconds Vbucket 199 2016-04-07T03:50:03.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49110 Milliseconds Vbucket 199 2016-04-07T03:50:03.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49140 Milliseconds Vbucket 199 2016-04-07T03:50:03.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49170 Milliseconds Vbucket 199 2016-04-07T03:50:03.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49200 Milliseconds Vbucket 199 2016-04-07T03:50:03.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49230 Milliseconds Vbucket 199 2016-04-07T03:50:03.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49260 Milliseconds Vbucket 199 2016-04-07T03:50:03.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49290 Milliseconds Vbucket 199 2016-04-07T03:50:03.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49320 Milliseconds Vbucket 199 2016-04-07T03:50:03.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49350 Milliseconds Vbucket 199 2016-04-07T03:50:03.715-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.536µs CommitTime 549.910867ms TotalFlushTime 9m30.286366252s TotalCommitTime 38.327159731s 2016-04-07T03:50:03.715-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 14325971, 14325971, 106 committed:true 2016-04-07T03:50:03.715-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 14325971, 14325971, 106 committed:true) SnapCreateDur 559.983888ms SnapOpenDur 32.567µs 2016-04-07T03:50:03.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49380 Milliseconds Vbucket 199 2016-04-07T03:50:03.752-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:03.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49410 Milliseconds Vbucket 199 2016-04-07T03:50:03.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49440 Milliseconds Vbucket 199 2016-04-07T03:50:03.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49470 Milliseconds Vbucket 199 2016-04-07T03:50:03.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49500 Milliseconds Vbucket 199 2016-04-07T03:50:03.885-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14330000 QueuedCount 50000 2016-04-07T03:50:03.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49530 Milliseconds Vbucket 199 2016-04-07T03:50:03.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49560 Milliseconds Vbucket 199 2016-04-07T03:50:03.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49590 Milliseconds Vbucket 199 2016-04-07T03:50:03.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49620 Milliseconds Vbucket 199 2016-04-07T03:50:04.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49650 Milliseconds Vbucket 199 2016-04-07T03:50:04.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49680 Milliseconds Vbucket 199 2016-04-07T03:50:04.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49710 Milliseconds Vbucket 199 2016-04-07T03:50:04.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49740 Milliseconds Vbucket 199 2016-04-07T03:50:04.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49770 Milliseconds Vbucket 199 2016-04-07T03:50:04.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49800 Milliseconds Vbucket 199 2016-04-07T03:50:04.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49830 Milliseconds Vbucket 199 2016-04-07T03:50:04.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49860 Milliseconds Vbucket 199 2016-04-07T03:50:04.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49890 Milliseconds Vbucket 199 2016-04-07T03:50:04.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49920 Milliseconds Vbucket 199 2016-04-07T03:50:04.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49950 Milliseconds Vbucket 199 2016-04-07T03:50:04.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 49980 Milliseconds Vbucket 199 2016-04-07T03:50:04.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50010 Milliseconds Vbucket 199 2016-04-07T03:50:04.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50040 Milliseconds Vbucket 199 2016-04-07T03:50:04.414-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14340000 QueuedCount 50000 2016-04-07T03:50:04.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50070 Milliseconds Vbucket 199 2016-04-07T03:50:04.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50100 Milliseconds Vbucket 199 2016-04-07T03:50:04.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50130 Milliseconds Vbucket 199 2016-04-07T03:50:04.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50160 Milliseconds Vbucket 199 2016-04-07T03:50:04.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50190 Milliseconds Vbucket 199 2016-04-07T03:50:04.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50220 Milliseconds Vbucket 199 2016-04-07T03:50:04.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50250 Milliseconds Vbucket 199 2016-04-07T03:50:04.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50280 Milliseconds Vbucket 199 2016-04-07T03:50:04.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50310 Milliseconds Vbucket 199 2016-04-07T03:50:04.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50340 Milliseconds Vbucket 199 2016-04-07T03:50:04.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50370 Milliseconds Vbucket 199 2016-04-07T03:50:04.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50400 Milliseconds Vbucket 199 2016-04-07T03:50:04.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50430 Milliseconds Vbucket 199 2016-04-07T03:50:04.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50460 Milliseconds Vbucket 199 2016-04-07T03:50:04.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50490 Milliseconds Vbucket 199 2016-04-07T03:50:04.852-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:04.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50520 Milliseconds Vbucket 199 2016-04-07T03:50:04.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50550 Milliseconds Vbucket 199 2016-04-07T03:50:04.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50580 Milliseconds Vbucket 199 2016-04-07T03:50:04.947-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14350000 QueuedCount 50000 2016-04-07T03:50:04.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50610 Milliseconds Vbucket 199 2016-04-07T03:50:05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50640 Milliseconds Vbucket 199 2016-04-07T03:50:05.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50670 Milliseconds Vbucket 199 2016-04-07T03:50:05.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50700 Milliseconds Vbucket 199 2016-04-07T03:50:05.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50730 Milliseconds Vbucket 199 2016-04-07T03:50:05.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50760 Milliseconds Vbucket 199 2016-04-07T03:50:05.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50790 Milliseconds Vbucket 199 2016-04-07T03:50:05.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50820 Milliseconds Vbucket 199 2016-04-07T03:50:05.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50850 Milliseconds Vbucket 199 2016-04-07T03:50:05.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50880 Milliseconds Vbucket 199 2016-04-07T03:50:05.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50910 Milliseconds Vbucket 199 2016-04-07T03:50:05.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50940 Milliseconds Vbucket 199 2016-04-07T03:50:05.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 50970 Milliseconds Vbucket 199 2016-04-07T03:50:05.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51000 Milliseconds Vbucket 199 2016-04-07T03:50:05.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51030 Milliseconds Vbucket 199 2016-04-07T03:50:05.397-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14360000 QueuedCount 50000 2016-04-07T03:50:05.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51060 Milliseconds Vbucket 199 2016-04-07T03:50:05.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51090 Milliseconds Vbucket 199 2016-04-07T03:50:05.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51120 Milliseconds Vbucket 199 2016-04-07T03:50:05.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51150 Milliseconds Vbucket 199 2016-04-07T03:50:05.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51180 Milliseconds Vbucket 199 2016-04-07T03:50:05.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51210 Milliseconds Vbucket 199 2016-04-07T03:50:05.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51240 Milliseconds Vbucket 199 2016-04-07T03:50:05.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51270 Milliseconds Vbucket 199 2016-04-07T03:50:05.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51300 Milliseconds Vbucket 199 2016-04-07T03:50:05.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51330 Milliseconds Vbucket 199 2016-04-07T03:50:05.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51360 Milliseconds Vbucket 199 2016-04-07T03:50:05.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51390 Milliseconds Vbucket 199 2016-04-07T03:50:05.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51420 Milliseconds Vbucket 199 2016-04-07T03:50:05.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51450 Milliseconds Vbucket 199 2016-04-07T03:50:05.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51480 Milliseconds Vbucket 199 2016-04-07T03:50:05.844-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14370000 QueuedCount 50000 2016-04-07T03:50:05.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51510 Milliseconds Vbucket 199 2016-04-07T03:50:05.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51540 Milliseconds Vbucket 199 2016-04-07T03:50:05.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51570 Milliseconds Vbucket 199 2016-04-07T03:50:05.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51600 Milliseconds Vbucket 199 2016-04-07T03:50:05.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51630 Milliseconds Vbucket 199 2016-04-07T03:50:06.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51660 Milliseconds Vbucket 199 2016-04-07T03:50:06.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51690 Milliseconds Vbucket 199 2016-04-07T03:50:06.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51720 Milliseconds Vbucket 199 2016-04-07T03:50:06.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51750 Milliseconds Vbucket 199 2016-04-07T03:50:06.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51780 Milliseconds Vbucket 199 2016-04-07T03:50:06.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51810 Milliseconds Vbucket 199 2016-04-07T03:50:06.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51840 Milliseconds Vbucket 199 2016-04-07T03:50:06.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51870 Milliseconds Vbucket 199 2016-04-07T03:50:06.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51900 Milliseconds Vbucket 199 2016-04-07T03:50:06.288-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14380000 QueuedCount 47179 2016-04-07T03:50:06.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51930 Milliseconds Vbucket 199 2016-04-07T03:50:06.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51960 Milliseconds Vbucket 199 2016-04-07T03:50:06.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 51990 Milliseconds Vbucket 199 2016-04-07T03:50:06.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52020 Milliseconds Vbucket 199 2016-04-07T03:50:06.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52050 Milliseconds Vbucket 199 2016-04-07T03:50:06.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52080 Milliseconds Vbucket 199 2016-04-07T03:50:06.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52110 Milliseconds Vbucket 199 2016-04-07T03:50:06.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52140 Milliseconds Vbucket 199 2016-04-07T03:50:06.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52170 Milliseconds Vbucket 199 2016-04-07T03:50:06.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52200 Milliseconds Vbucket 199 2016-04-07T03:50:06.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52230 Milliseconds Vbucket 199 2016-04-07T03:50:06.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52260 Milliseconds Vbucket 199 2016-04-07T03:50:06.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52290 Milliseconds Vbucket 199 2016-04-07T03:50:06.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52320 Milliseconds Vbucket 199 2016-04-07T03:50:06.698-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14390000 QueuedCount 37179 2016-04-07T03:50:06.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52350 Milliseconds Vbucket 199 2016-04-07T03:50:06.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52380 Milliseconds Vbucket 199 2016-04-07T03:50:06.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52410 Milliseconds Vbucket 199 2016-04-07T03:50:06.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52440 Milliseconds Vbucket 199 2016-04-07T03:50:06.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52470 Milliseconds Vbucket 199 2016-04-07T03:50:06.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52500 Milliseconds Vbucket 199 2016-04-07T03:50:06.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52530 Milliseconds Vbucket 199 2016-04-07T03:50:06.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52560 Milliseconds Vbucket 199 2016-04-07T03:50:06.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52590 Milliseconds Vbucket 199 2016-04-07T03:50:06.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52620 Milliseconds Vbucket 199 2016-04-07T03:50:07.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52650 Milliseconds Vbucket 199 2016-04-07T03:50:07.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52680 Milliseconds Vbucket 199 2016-04-07T03:50:07.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52710 Milliseconds Vbucket 199 2016-04-07T03:50:07.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52740 Milliseconds Vbucket 199 2016-04-07T03:50:07.105-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14400000 QueuedCount 27179 2016-04-07T03:50:07.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52770 Milliseconds Vbucket 199 2016-04-07T03:50:07.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52800 Milliseconds Vbucket 199 2016-04-07T03:50:07.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52830 Milliseconds Vbucket 199 2016-04-07T03:50:07.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52860 Milliseconds Vbucket 199 2016-04-07T03:50:07.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52890 Milliseconds Vbucket 199 2016-04-07T03:50:07.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52920 Milliseconds Vbucket 199 2016-04-07T03:50:07.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52950 Milliseconds Vbucket 199 2016-04-07T03:50:07.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 52980 Milliseconds Vbucket 199 2016-04-07T03:50:07.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53010 Milliseconds Vbucket 199 2016-04-07T03:50:07.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53040 Milliseconds Vbucket 199 2016-04-07T03:50:07.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53070 Milliseconds Vbucket 199 2016-04-07T03:50:07.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53100 Milliseconds Vbucket 199 2016-04-07T03:50:07.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53130 Milliseconds Vbucket 199 2016-04-07T03:50:07.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53160 Milliseconds Vbucket 199 2016-04-07T03:50:07.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53190 Milliseconds Vbucket 199 2016-04-07T03:50:07.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53220 Milliseconds Vbucket 199 2016-04-07T03:50:07.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53250 Milliseconds Vbucket 199 2016-04-07T03:50:07.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53280 Milliseconds Vbucket 199 2016-04-07T03:50:07.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53310 Milliseconds Vbucket 199 2016-04-07T03:50:07.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53340 Milliseconds Vbucket 199 2016-04-07T03:50:07.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53370 Milliseconds Vbucket 199 2016-04-07T03:50:07.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53400 Milliseconds Vbucket 199 2016-04-07T03:50:07.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53430 Milliseconds Vbucket 199 2016-04-07T03:50:07.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53460 Milliseconds Vbucket 199 2016-04-07T03:50:07.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53490 Milliseconds Vbucket 199 2016-04-07T03:50:07.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53520 Milliseconds Vbucket 199 2016-04-07T03:50:07.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53550 Milliseconds Vbucket 199 2016-04-07T03:50:07.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53580 Milliseconds Vbucket 199 2016-04-07T03:50:07.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53610 Milliseconds Vbucket 199 2016-04-07T03:50:08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53640 Milliseconds Vbucket 199 2016-04-07T03:50:08.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53670 Milliseconds Vbucket 199 2016-04-07T03:50:08.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53700 Milliseconds Vbucket 199 2016-04-07T03:50:08.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53730 Milliseconds Vbucket 199 2016-04-07T03:50:08.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53760 Milliseconds Vbucket 199 2016-04-07T03:50:08.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53790 Milliseconds Vbucket 199 2016-04-07T03:50:08.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53820 Milliseconds Vbucket 199 2016-04-07T03:50:08.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53850 Milliseconds Vbucket 199 2016-04-07T03:50:08.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53880 Milliseconds Vbucket 199 2016-04-07T03:50:08.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53910 Milliseconds Vbucket 199 2016-04-07T03:50:08.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53940 Milliseconds Vbucket 199 2016-04-07T03:50:08.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 53970 Milliseconds Vbucket 199 2016-04-07T03:50:08.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54000 Milliseconds Vbucket 199 2016-04-07T03:50:08.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54030 Milliseconds Vbucket 199 2016-04-07T03:50:08.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54060 Milliseconds Vbucket 199 2016-04-07T03:50:08.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54090 Milliseconds Vbucket 199 2016-04-07T03:50:08.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54120 Milliseconds Vbucket 199 2016-04-07T03:50:08.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54150 Milliseconds Vbucket 199 2016-04-07T03:50:08.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54180 Milliseconds Vbucket 199 2016-04-07T03:50:08.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54210 Milliseconds Vbucket 199 2016-04-07T03:50:08.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54240 Milliseconds Vbucket 199 2016-04-07T03:50:08.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54270 Milliseconds Vbucket 199 2016-04-07T03:50:08.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54300 Milliseconds Vbucket 199 2016-04-07T03:50:08.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54330 Milliseconds Vbucket 199 2016-04-07T03:50:08.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54360 Milliseconds Vbucket 199 2016-04-07T03:50:08.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54390 Milliseconds Vbucket 199 2016-04-07T03:50:08.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54420 Milliseconds Vbucket 199 2016-04-07T03:50:08.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54450 Milliseconds Vbucket 199 2016-04-07T03:50:08.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54480 Milliseconds Vbucket 199 2016-04-07T03:50:08.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54510 Milliseconds Vbucket 199 2016-04-07T03:50:08.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54540 Milliseconds Vbucket 199 2016-04-07T03:50:08.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54570 Milliseconds Vbucket 199 2016-04-07T03:50:08.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54600 Milliseconds Vbucket 199 2016-04-07T03:50:08.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54630 Milliseconds Vbucket 199 2016-04-07T03:50:09.004-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14410000 QueuedCount 17179 2016-04-07T03:50:09.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54660 Milliseconds Vbucket 199 2016-04-07T03:50:09.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54690 Milliseconds Vbucket 199 2016-04-07T03:50:09.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54720 Milliseconds Vbucket 199 2016-04-07T03:50:09.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54750 Milliseconds Vbucket 199 2016-04-07T03:50:09.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54780 Milliseconds Vbucket 199 2016-04-07T03:50:09.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54810 Milliseconds Vbucket 199 2016-04-07T03:50:09.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54840 Milliseconds Vbucket 199 2016-04-07T03:50:09.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54870 Milliseconds Vbucket 199 2016-04-07T03:50:09.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54900 Milliseconds Vbucket 199 2016-04-07T03:50:09.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54930 Milliseconds Vbucket 199 2016-04-07T03:50:09.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54960 Milliseconds Vbucket 199 2016-04-07T03:50:09.321-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14420000 QueuedCount 7179 2016-04-07T03:50:09.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 54990 Milliseconds Vbucket 199 2016-04-07T03:50:09.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55020 Milliseconds Vbucket 199 2016-04-07T03:50:09.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55050 Milliseconds Vbucket 199 2016-04-07T03:50:09.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55080 Milliseconds Vbucket 199 2016-04-07T03:50:09.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55110 Milliseconds Vbucket 199 2016-04-07T03:50:09.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55140 Milliseconds Vbucket 199 2016-04-07T03:50:09.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55170 Milliseconds Vbucket 199 2016-04-07T03:50:09.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55200 Milliseconds Vbucket 199 2016-04-07T03:50:09.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55230 Milliseconds Vbucket 199 2016-04-07T03:50:09.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55260 Milliseconds Vbucket 199 2016-04-07T03:50:09.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55290 Milliseconds Vbucket 199 2016-04-07T03:50:09.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55320 Milliseconds Vbucket 199 2016-04-07T03:50:09.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55350 Milliseconds Vbucket 199 2016-04-07T03:50:09.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55380 Milliseconds Vbucket 199 2016-04-07T03:50:09.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55410 Milliseconds Vbucket 199 2016-04-07T03:50:09.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55440 Milliseconds Vbucket 199 2016-04-07T03:50:09.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55470 Milliseconds Vbucket 199 2016-04-07T03:50:09.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55500 Milliseconds Vbucket 199 2016-04-07T03:50:09.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55530 Milliseconds Vbucket 199 2016-04-07T03:50:09.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55560 Milliseconds Vbucket 199 2016-04-07T03:50:09.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55590 Milliseconds Vbucket 199 2016-04-07T03:50:09.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55620 Milliseconds Vbucket 199 2016-04-07T03:50:10.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55650 Milliseconds Vbucket 199 2016-04-07T03:50:10.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55680 Milliseconds Vbucket 199 2016-04-07T03:50:10.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55710 Milliseconds Vbucket 199 2016-04-07T03:50:10.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55740 Milliseconds Vbucket 199 2016-04-07T03:50:10.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55770 Milliseconds Vbucket 199 2016-04-07T03:50:10.159-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.641µs CommitTime 468.666576ms TotalFlushTime 9m36.085397075s TotalCommitTime 38.795826307s 2016-04-07T03:50:10.159-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 14427179, 14427179, 107 committed:true 2016-04-07T03:50:10.159-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 14427179, 14427179, 107 committed:true) SnapCreateDur 479.014113ms SnapOpenDur 37.816µs 2016-04-07T03:50:10.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55800 Milliseconds Vbucket 199 2016-04-07T03:50:10.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55830 Milliseconds Vbucket 199 2016-04-07T03:50:10.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55860 Milliseconds Vbucket 199 2016-04-07T03:50:10.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55890 Milliseconds Vbucket 199 2016-04-07T03:50:10.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55920 Milliseconds Vbucket 199 2016-04-07T03:50:10.29-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14430000 QueuedCount 25767 2016-04-07T03:50:10.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55950 Milliseconds Vbucket 199 2016-04-07T03:50:10.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 55980 Milliseconds Vbucket 199 2016-04-07T03:50:10.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56010 Milliseconds Vbucket 199 2016-04-07T03:50:10.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56040 Milliseconds Vbucket 199 2016-04-07T03:50:10.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56070 Milliseconds Vbucket 199 2016-04-07T03:50:10.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56100 Milliseconds Vbucket 199 2016-04-07T03:50:10.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56130 Milliseconds Vbucket 199 2016-04-07T03:50:10.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56160 Milliseconds Vbucket 199 2016-04-07T03:50:10.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56190 Milliseconds Vbucket 199 2016-04-07T03:50:10.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56220 Milliseconds Vbucket 199 2016-04-07T03:50:10.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56250 Milliseconds Vbucket 199 2016-04-07T03:50:10.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56280 Milliseconds Vbucket 199 2016-04-07T03:50:10.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56310 Milliseconds Vbucket 199 2016-04-07T03:50:10.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56340 Milliseconds Vbucket 199 2016-04-07T03:50:10.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56370 Milliseconds Vbucket 199 2016-04-07T03:50:10.746-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14440000 QueuedCount 15767 2016-04-07T03:50:10.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56400 Milliseconds Vbucket 199 2016-04-07T03:50:10.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56430 Milliseconds Vbucket 199 2016-04-07T03:50:10.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56460 Milliseconds Vbucket 199 2016-04-07T03:50:10.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56490 Milliseconds Vbucket 199 2016-04-07T03:50:10.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56520 Milliseconds Vbucket 199 2016-04-07T03:50:10.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56550 Milliseconds Vbucket 199 2016-04-07T03:50:10.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56580 Milliseconds Vbucket 199 2016-04-07T03:50:10.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56610 Milliseconds Vbucket 199 2016-04-07T03:50:11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56640 Milliseconds Vbucket 199 2016-04-07T03:50:11.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56670 Milliseconds Vbucket 199 2016-04-07T03:50:11.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56700 Milliseconds Vbucket 199 2016-04-07T03:50:11.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56730 Milliseconds Vbucket 199 2016-04-07T03:50:11.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56760 Milliseconds Vbucket 199 2016-04-07T03:50:11.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56790 Milliseconds Vbucket 199 2016-04-07T03:50:11.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56820 Milliseconds Vbucket 199 2016-04-07T03:50:11.19-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14450000 QueuedCount 5767 2016-04-07T03:50:11.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56850 Milliseconds Vbucket 199 2016-04-07T03:50:11.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56880 Milliseconds Vbucket 199 2016-04-07T03:50:11.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56910 Milliseconds Vbucket 199 2016-04-07T03:50:11.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56940 Milliseconds Vbucket 199 2016-04-07T03:50:11.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 56970 Milliseconds Vbucket 199 2016-04-07T03:50:11.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57000 Milliseconds Vbucket 199 2016-04-07T03:50:11.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57030 Milliseconds Vbucket 199 2016-04-07T03:50:11.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57060 Milliseconds Vbucket 199 2016-04-07T03:50:11.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57090 Milliseconds Vbucket 199 2016-04-07T03:50:11.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57120 Milliseconds Vbucket 199 2016-04-07T03:50:11.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57150 Milliseconds Vbucket 199 2016-04-07T03:50:11.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57180 Milliseconds Vbucket 199 2016-04-07T03:50:11.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57210 Milliseconds Vbucket 199 2016-04-07T03:50:11.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57240 Milliseconds Vbucket 199 2016-04-07T03:50:11.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57270 Milliseconds Vbucket 199 2016-04-07T03:50:11.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57300 Milliseconds Vbucket 199 2016-04-07T03:50:11.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57330 Milliseconds Vbucket 199 2016-04-07T03:50:11.701-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 14.404µs CommitTime 317.91497ms TotalFlushTime 9m37.262918211s TotalCommitTime 39.113741277s 2016-04-07T03:50:11.701-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 14455767, 14455767, 108 committed:true 2016-04-07T03:50:11.701-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 14455767, 14455767, 108 committed:true) SnapCreateDur 328.38932ms SnapOpenDur 39.074µs 2016-04-07T03:50:11.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57360 Milliseconds Vbucket 199 2016-04-07T03:50:11.734-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:11.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57390 Milliseconds Vbucket 199 2016-04-07T03:50:11.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57420 Milliseconds Vbucket 199 2016-04-07T03:50:11.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57450 Milliseconds Vbucket 199 2016-04-07T03:50:11.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57480 Milliseconds Vbucket 199 2016-04-07T03:50:11.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57510 Milliseconds Vbucket 199 2016-04-07T03:50:11.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57540 Milliseconds Vbucket 199 2016-04-07T03:50:11.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57570 Milliseconds Vbucket 199 2016-04-07T03:50:11.947-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14460000 QueuedCount 50000 2016-04-07T03:50:11.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57600 Milliseconds Vbucket 199 2016-04-07T03:50:11.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57630 Milliseconds Vbucket 199 2016-04-07T03:50:12.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57660 Milliseconds Vbucket 199 2016-04-07T03:50:12.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57690 Milliseconds Vbucket 199 2016-04-07T03:50:12.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57720 Milliseconds Vbucket 199 2016-04-07T03:50:12.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57750 Milliseconds Vbucket 199 2016-04-07T03:50:12.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57780 Milliseconds Vbucket 199 2016-04-07T03:50:12.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57810 Milliseconds Vbucket 199 2016-04-07T03:50:12.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57840 Milliseconds Vbucket 199 2016-04-07T03:50:12.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57870 Milliseconds Vbucket 199 2016-04-07T03:50:12.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57900 Milliseconds Vbucket 199 2016-04-07T03:50:12.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57930 Milliseconds Vbucket 199 2016-04-07T03:50:12.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57960 Milliseconds Vbucket 199 2016-04-07T03:50:12.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 57990 Milliseconds Vbucket 199 2016-04-07T03:50:12.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58020 Milliseconds Vbucket 199 2016-04-07T03:50:12.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58050 Milliseconds Vbucket 199 2016-04-07T03:50:12.411-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14470000 QueuedCount 50000 2016-04-07T03:50:12.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58080 Milliseconds Vbucket 199 2016-04-07T03:50:12.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58110 Milliseconds Vbucket 199 2016-04-07T03:50:12.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58140 Milliseconds Vbucket 199 2016-04-07T03:50:12.504-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:12.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58170 Milliseconds Vbucket 199 2016-04-07T03:50:12.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58200 Milliseconds Vbucket 199 2016-04-07T03:50:12.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58230 Milliseconds Vbucket 199 2016-04-07T03:50:12.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58260 Milliseconds Vbucket 199 2016-04-07T03:50:12.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58290 Milliseconds Vbucket 199 2016-04-07T03:50:12.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58320 Milliseconds Vbucket 199 2016-04-07T03:50:12.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58350 Milliseconds Vbucket 199 2016-04-07T03:50:12.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58380 Milliseconds Vbucket 199 2016-04-07T03:50:12.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58410 Milliseconds Vbucket 199 2016-04-07T03:50:12.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58440 Milliseconds Vbucket 199 2016-04-07T03:50:12.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58470 Milliseconds Vbucket 199 2016-04-07T03:50:12.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58500 Milliseconds Vbucket 199 2016-04-07T03:50:12.878-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14480000 QueuedCount 50000 2016-04-07T03:50:12.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58530 Milliseconds Vbucket 199 2016-04-07T03:50:12.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58560 Milliseconds Vbucket 199 2016-04-07T03:50:12.944-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":5469330,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3826076477,"bucket-1:myindex:avg_ts_items_count":67860,"bucket-1:myindex:build_progress":72,"bucket-1:myindex:data_size":4674287541,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":26904571904,"bucket-1:myindex:flush_queue_size":50034,"bucket-1:myindex:frag_percent":82,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":1516144307,"bucket-1:myindex:items_count":14427179,"bucket-1:myindex:num_commits":108,"bucket-1:myindex:num_compactions":2,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":14427179,"bucket-1:myindex:num_docs_pending":319,"bucket-1:myindex:num_docs_queued":5572502,"bucket-1:myindex:num_flush_queued":14530382,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":108,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1541781290,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"108 39113741277 -2494905484249313387","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"14480349 92567937073 610827907522054069","bucket-1:myindex:timings/storage_handle_open":"108 2580646 64117921512","bucket-1:myindex:timings/storage_info":"433 4142362 108525017592","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"109 72919502 730963751483752","bucket-1:myindex:timings/storage_meta_set":"108 370552022 107832313775568862","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"28960697 476416909562 318313253296252736","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":19999679,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":448,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"2484 307754219 52088453297999"} 2016-04-07T03:50:12.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58590 Milliseconds Vbucket 199 2016-04-07T03:50:12.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58620 Milliseconds Vbucket 199 2016-04-07T03:50:13.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58650 Milliseconds Vbucket 199 2016-04-07T03:50:13.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58680 Milliseconds Vbucket 199 2016-04-07T03:50:13.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58710 Milliseconds Vbucket 199 2016-04-07T03:50:13.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58740 Milliseconds Vbucket 199 2016-04-07T03:50:13.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58770 Milliseconds Vbucket 199 2016-04-07T03:50:13.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58800 Milliseconds Vbucket 199 2016-04-07T03:50:13.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58830 Milliseconds Vbucket 199 2016-04-07T03:50:13.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58860 Milliseconds Vbucket 199 2016-04-07T03:50:13.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58890 Milliseconds Vbucket 199 2016-04-07T03:50:13.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58920 Milliseconds Vbucket 199 2016-04-07T03:50:13.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58950 Milliseconds Vbucket 199 2016-04-07T03:50:13.322-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14490000 QueuedCount 50000 2016-04-07T03:50:13.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 58980 Milliseconds Vbucket 199 2016-04-07T03:50:13.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59010 Milliseconds Vbucket 199 2016-04-07T03:50:13.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59040 Milliseconds Vbucket 199 2016-04-07T03:50:13.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59070 Milliseconds Vbucket 199 2016-04-07T03:50:13.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59100 Milliseconds Vbucket 199 2016-04-07T03:50:13.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59130 Milliseconds Vbucket 199 2016-04-07T03:50:13.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59160 Milliseconds Vbucket 199 2016-04-07T03:50:13.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59190 Milliseconds Vbucket 199 2016-04-07T03:50:13.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59220 Milliseconds Vbucket 199 2016-04-07T03:50:13.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59250 Milliseconds Vbucket 199 2016-04-07T03:50:13.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59280 Milliseconds Vbucket 199 2016-04-07T03:50:13.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59310 Milliseconds Vbucket 199 2016-04-07T03:50:13.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59340 Milliseconds Vbucket 199 2016-04-07T03:50:13.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59370 Milliseconds Vbucket 199 2016-04-07T03:50:13.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59400 Milliseconds Vbucket 199 2016-04-07T03:50:13.769-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14500000 QueuedCount 50000 2016-04-07T03:50:13.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59430 Milliseconds Vbucket 199 2016-04-07T03:50:13.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59460 Milliseconds Vbucket 199 2016-04-07T03:50:13.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59490 Milliseconds Vbucket 199 2016-04-07T03:50:13.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59520 Milliseconds Vbucket 199 2016-04-07T03:50:13.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59550 Milliseconds Vbucket 199 2016-04-07T03:50:13.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59580 Milliseconds Vbucket 199 2016-04-07T03:50:13.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59610 Milliseconds Vbucket 199 2016-04-07T03:50:14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59640 Milliseconds Vbucket 199 2016-04-07T03:50:14.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59670 Milliseconds Vbucket 199 2016-04-07T03:50:14.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59700 Milliseconds Vbucket 199 2016-04-07T03:50:14.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59730 Milliseconds Vbucket 199 2016-04-07T03:50:14.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59760 Milliseconds Vbucket 199 2016-04-07T03:50:14.136-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:14.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59790 Milliseconds Vbucket 199 2016-04-07T03:50:14.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59820 Milliseconds Vbucket 199 2016-04-07T03:50:14.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59850 Milliseconds Vbucket 199 2016-04-07T03:50:14.223-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14510000 QueuedCount 50000 2016-04-07T03:50:14.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59880 Milliseconds Vbucket 199 2016-04-07T03:50:14.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59910 Milliseconds Vbucket 199 2016-04-07T03:50:14.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59940 Milliseconds Vbucket 199 2016-04-07T03:50:14.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 59970 Milliseconds Vbucket 199 2016-04-07T03:50:14.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60000 Milliseconds Vbucket 199 2016-04-07T03:50:14.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60030 Milliseconds Vbucket 199 2016-04-07T03:50:14.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60060 Milliseconds Vbucket 199 2016-04-07T03:50:14.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60090 Milliseconds Vbucket 199 2016-04-07T03:50:14.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60120 Milliseconds Vbucket 199 2016-04-07T03:50:14.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60150 Milliseconds Vbucket 199 2016-04-07T03:50:14.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60180 Milliseconds Vbucket 199 2016-04-07T03:50:14.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60210 Milliseconds Vbucket 199 2016-04-07T03:50:14.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60240 Milliseconds Vbucket 199 2016-04-07T03:50:14.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60270 Milliseconds Vbucket 199 2016-04-07T03:50:14.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60300 Milliseconds Vbucket 199 2016-04-07T03:50:14.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60330 Milliseconds Vbucket 199 2016-04-07T03:50:14.704-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14520000 QueuedCount 50000 2016-04-07T03:50:14.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60360 Milliseconds Vbucket 199 2016-04-07T03:50:14.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60390 Milliseconds Vbucket 199 2016-04-07T03:50:14.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60420 Milliseconds Vbucket 199 2016-04-07T03:50:14.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60450 Milliseconds Vbucket 199 2016-04-07T03:50:14.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60480 Milliseconds Vbucket 199 2016-04-07T03:50:14.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60510 Milliseconds Vbucket 199 2016-04-07T03:50:14.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60540 Milliseconds Vbucket 199 2016-04-07T03:50:14.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60570 Milliseconds Vbucket 199 2016-04-07T03:50:14.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60600 Milliseconds Vbucket 199 2016-04-07T03:50:14.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60630 Milliseconds Vbucket 199 2016-04-07T03:50:15.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60660 Milliseconds Vbucket 199 2016-04-07T03:50:15.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60690 Milliseconds Vbucket 199 2016-04-07T03:50:15.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60720 Milliseconds Vbucket 199 2016-04-07T03:50:15.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60750 Milliseconds Vbucket 199 2016-04-07T03:50:15.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60780 Milliseconds Vbucket 199 2016-04-07T03:50:15.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60810 Milliseconds Vbucket 199 2016-04-07T03:50:15.191-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14530000 QueuedCount 50000 2016-04-07T03:50:15.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60840 Milliseconds Vbucket 199 2016-04-07T03:50:15.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60870 Milliseconds Vbucket 199 2016-04-07T03:50:15.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60900 Milliseconds Vbucket 199 2016-04-07T03:50:15.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60930 Milliseconds Vbucket 199 2016-04-07T03:50:15.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60960 Milliseconds Vbucket 199 2016-04-07T03:50:15.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 60990 Milliseconds Vbucket 199 2016-04-07T03:50:15.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61020 Milliseconds Vbucket 199 2016-04-07T03:50:15.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61050 Milliseconds Vbucket 199 2016-04-07T03:50:15.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61080 Milliseconds Vbucket 199 2016-04-07T03:50:15.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61110 Milliseconds Vbucket 199 2016-04-07T03:50:15.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61140 Milliseconds Vbucket 199 2016-04-07T03:50:15.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61170 Milliseconds Vbucket 199 2016-04-07T03:50:15.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61200 Milliseconds Vbucket 199 2016-04-07T03:50:15.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61230 Milliseconds Vbucket 199 2016-04-07T03:50:15.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61260 Milliseconds Vbucket 199 2016-04-07T03:50:15.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61290 Milliseconds Vbucket 199 2016-04-07T03:50:15.675-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14540000 QueuedCount 50000 2016-04-07T03:50:15.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61320 Milliseconds Vbucket 199 2016-04-07T03:50:15.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61350 Milliseconds Vbucket 199 2016-04-07T03:50:15.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61380 Milliseconds Vbucket 199 2016-04-07T03:50:15.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61410 Milliseconds Vbucket 199 2016-04-07T03:50:15.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61440 Milliseconds Vbucket 199 2016-04-07T03:50:15.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61470 Milliseconds Vbucket 199 2016-04-07T03:50:15.848-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:15.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61500 Milliseconds Vbucket 199 2016-04-07T03:50:15.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61530 Milliseconds Vbucket 199 2016-04-07T03:50:15.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61560 Milliseconds Vbucket 199 2016-04-07T03:50:15.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61590 Milliseconds Vbucket 199 2016-04-07T03:50:15.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61620 Milliseconds Vbucket 199 2016-04-07T03:50:16.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61650 Milliseconds Vbucket 199 2016-04-07T03:50:16.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61680 Milliseconds Vbucket 199 2016-04-07T03:50:16.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61710 Milliseconds Vbucket 199 2016-04-07T03:50:16.095-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14550000 QueuedCount 50000 2016-04-07T03:50:16.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61740 Milliseconds Vbucket 199 2016-04-07T03:50:16.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61770 Milliseconds Vbucket 199 2016-04-07T03:50:16.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61800 Milliseconds Vbucket 199 2016-04-07T03:50:16.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61830 Milliseconds Vbucket 199 2016-04-07T03:50:16.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61860 Milliseconds Vbucket 199 2016-04-07T03:50:16.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61890 Milliseconds Vbucket 199 2016-04-07T03:50:16.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61920 Milliseconds Vbucket 199 2016-04-07T03:50:16.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61950 Milliseconds Vbucket 199 2016-04-07T03:50:16.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 61980 Milliseconds Vbucket 199 2016-04-07T03:50:16.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62010 Milliseconds Vbucket 199 2016-04-07T03:50:16.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62040 Milliseconds Vbucket 199 2016-04-07T03:50:16.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62070 Milliseconds Vbucket 199 2016-04-07T03:50:16.459-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14560000 QueuedCount 50000 2016-04-07T03:50:16.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62100 Milliseconds Vbucket 199 2016-04-07T03:50:16.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62130 Milliseconds Vbucket 199 2016-04-07T03:50:16.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62160 Milliseconds Vbucket 199 2016-04-07T03:50:16.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62190 Milliseconds Vbucket 199 2016-04-07T03:50:16.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62220 Milliseconds Vbucket 199 2016-04-07T03:50:16.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62250 Milliseconds Vbucket 199 2016-04-07T03:50:16.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62280 Milliseconds Vbucket 199 2016-04-07T03:50:16.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62310 Milliseconds Vbucket 199 2016-04-07T03:50:16.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62340 Milliseconds Vbucket 199 2016-04-07T03:50:16.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62370 Milliseconds Vbucket 199 2016-04-07T03:50:16.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62400 Milliseconds Vbucket 199 2016-04-07T03:50:16.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62430 Milliseconds Vbucket 199 2016-04-07T03:50:16.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62460 Milliseconds Vbucket 199 2016-04-07T03:50:16.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62490 Milliseconds Vbucket 199 2016-04-07T03:50:16.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62520 Milliseconds Vbucket 199 2016-04-07T03:50:16.89-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14570000 QueuedCount 50000 2016-04-07T03:50:16.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62550 Milliseconds Vbucket 199 2016-04-07T03:50:16.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62580 Milliseconds Vbucket 199 2016-04-07T03:50:16.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62610 Milliseconds Vbucket 199 2016-04-07T03:50:17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62640 Milliseconds Vbucket 199 2016-04-07T03:50:17.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62670 Milliseconds Vbucket 199 2016-04-07T03:50:17.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62700 Milliseconds Vbucket 199 2016-04-07T03:50:17.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62730 Milliseconds Vbucket 199 2016-04-07T03:50:17.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62760 Milliseconds Vbucket 199 2016-04-07T03:50:17.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62790 Milliseconds Vbucket 199 2016-04-07T03:50:17.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62820 Milliseconds Vbucket 199 2016-04-07T03:50:17.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62850 Milliseconds Vbucket 199 2016-04-07T03:50:17.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62880 Milliseconds Vbucket 199 2016-04-07T03:50:17.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62910 Milliseconds Vbucket 199 2016-04-07T03:50:17.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62940 Milliseconds Vbucket 199 2016-04-07T03:50:17.302-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14580000 QueuedCount 46781 2016-04-07T03:50:17.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 62970 Milliseconds Vbucket 199 2016-04-07T03:50:17.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63000 Milliseconds Vbucket 199 2016-04-07T03:50:17.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63030 Milliseconds Vbucket 199 2016-04-07T03:50:17.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63060 Milliseconds Vbucket 199 2016-04-07T03:50:17.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63090 Milliseconds Vbucket 199 2016-04-07T03:50:17.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63120 Milliseconds Vbucket 199 2016-04-07T03:50:17.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63150 Milliseconds Vbucket 199 2016-04-07T03:50:17.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63180 Milliseconds Vbucket 199 2016-04-07T03:50:17.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63210 Milliseconds Vbucket 199 2016-04-07T03:50:17.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63240 Milliseconds Vbucket 199 2016-04-07T03:50:17.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63270 Milliseconds Vbucket 199 2016-04-07T03:50:17.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63300 Milliseconds Vbucket 199 2016-04-07T03:50:17.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63330 Milliseconds Vbucket 199 2016-04-07T03:50:17.699-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14590000 QueuedCount 36781 2016-04-07T03:50:17.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63360 Milliseconds Vbucket 199 2016-04-07T03:50:17.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63390 Milliseconds Vbucket 199 2016-04-07T03:50:17.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63420 Milliseconds Vbucket 199 2016-04-07T03:50:17.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63450 Milliseconds Vbucket 199 2016-04-07T03:50:17.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63480 Milliseconds Vbucket 199 2016-04-07T03:50:17.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63510 Milliseconds Vbucket 199 2016-04-07T03:50:17.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63540 Milliseconds Vbucket 199 2016-04-07T03:50:17.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63570 Milliseconds Vbucket 199 2016-04-07T03:50:17.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63600 Milliseconds Vbucket 199 2016-04-07T03:50:17.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63630 Milliseconds Vbucket 199 2016-04-07T03:50:18.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63660 Milliseconds Vbucket 199 2016-04-07T03:50:18.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63690 Milliseconds Vbucket 199 2016-04-07T03:50:18.067-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14600000 QueuedCount 26781 2016-04-07T03:50:18.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63720 Milliseconds Vbucket 199 2016-04-07T03:50:18.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63750 Milliseconds Vbucket 199 2016-04-07T03:50:18.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63780 Milliseconds Vbucket 199 2016-04-07T03:50:18.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63810 Milliseconds Vbucket 199 2016-04-07T03:50:18.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63840 Milliseconds Vbucket 199 2016-04-07T03:50:18.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63870 Milliseconds Vbucket 199 2016-04-07T03:50:18.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63900 Milliseconds Vbucket 199 2016-04-07T03:50:18.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63930 Milliseconds Vbucket 199 2016-04-07T03:50:18.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63960 Milliseconds Vbucket 199 2016-04-07T03:50:18.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 63990 Milliseconds Vbucket 199 2016-04-07T03:50:18.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64020 Milliseconds Vbucket 199 2016-04-07T03:50:18.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64050 Milliseconds Vbucket 199 2016-04-07T03:50:18.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64080 Milliseconds Vbucket 199 2016-04-07T03:50:18.467-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14610000 QueuedCount 16781 2016-04-07T03:50:18.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64110 Milliseconds Vbucket 199 2016-04-07T03:50:18.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64140 Milliseconds Vbucket 199 2016-04-07T03:50:18.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64170 Milliseconds Vbucket 199 2016-04-07T03:50:18.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64200 Milliseconds Vbucket 199 2016-04-07T03:50:18.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64230 Milliseconds Vbucket 199 2016-04-07T03:50:18.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64260 Milliseconds Vbucket 199 2016-04-07T03:50:18.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64290 Milliseconds Vbucket 199 2016-04-07T03:50:18.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64320 Milliseconds Vbucket 199 2016-04-07T03:50:18.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64350 Milliseconds Vbucket 199 2016-04-07T03:50:18.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64380 Milliseconds Vbucket 199 2016-04-07T03:50:18.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64410 Milliseconds Vbucket 199 2016-04-07T03:50:18.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64440 Milliseconds Vbucket 199 2016-04-07T03:50:18.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64470 Milliseconds Vbucket 199 2016-04-07T03:50:18.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64500 Milliseconds Vbucket 199 2016-04-07T03:50:18.862-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14620000 QueuedCount 6781 2016-04-07T03:50:18.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64530 Milliseconds Vbucket 199 2016-04-07T03:50:18.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64560 Milliseconds Vbucket 199 2016-04-07T03:50:18.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64590 Milliseconds Vbucket 199 2016-04-07T03:50:18.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64620 Milliseconds Vbucket 199 2016-04-07T03:50:19.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64650 Milliseconds Vbucket 199 2016-04-07T03:50:19.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64680 Milliseconds Vbucket 199 2016-04-07T03:50:19.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64710 Milliseconds Vbucket 199 2016-04-07T03:50:19.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64740 Milliseconds Vbucket 199 2016-04-07T03:50:19.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64770 Milliseconds Vbucket 199 2016-04-07T03:50:19.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64800 Milliseconds Vbucket 199 2016-04-07T03:50:19.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64830 Milliseconds Vbucket 199 2016-04-07T03:50:19.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64860 Milliseconds Vbucket 199 2016-04-07T03:50:19.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64890 Milliseconds Vbucket 199 2016-04-07T03:50:19.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64920 Milliseconds Vbucket 199 2016-04-07T03:50:19.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64950 Milliseconds Vbucket 199 2016-04-07T03:50:19.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 64980 Milliseconds Vbucket 199 2016-04-07T03:50:19.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65010 Milliseconds Vbucket 199 2016-04-07T03:50:19.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65040 Milliseconds Vbucket 199 2016-04-07T03:50:19.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65070 Milliseconds Vbucket 199 2016-04-07T03:50:19.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65100 Milliseconds Vbucket 199 2016-04-07T03:50:19.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65130 Milliseconds Vbucket 199 2016-04-07T03:50:19.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65160 Milliseconds Vbucket 199 2016-04-07T03:50:19.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65190 Milliseconds Vbucket 199 2016-04-07T03:50:19.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65220 Milliseconds Vbucket 199 2016-04-07T03:50:19.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65250 Milliseconds Vbucket 199 2016-04-07T03:50:19.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65280 Milliseconds Vbucket 199 2016-04-07T03:50:19.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65310 Milliseconds Vbucket 199 2016-04-07T03:50:19.695-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.474µs CommitTime 581.008602ms TotalFlushTime 9m44.370228212s TotalCommitTime 39.694749879s 2016-04-07T03:50:19.695-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 14626781, 14626781, 109 committed:true 2016-04-07T03:50:19.695-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 14626781, 14626781, 109 committed:true) SnapCreateDur 591.570271ms SnapOpenDur 34.615µs 2016-04-07T03:50:19.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65340 Milliseconds Vbucket 199 2016-04-07T03:50:19.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65370 Milliseconds Vbucket 199 2016-04-07T03:50:19.73-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:19.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65400 Milliseconds Vbucket 199 2016-04-07T03:50:19.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65430 Milliseconds Vbucket 199 2016-04-07T03:50:19.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65460 Milliseconds Vbucket 199 2016-04-07T03:50:19.843-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14630000 QueuedCount 50000 2016-04-07T03:50:19.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65490 Milliseconds Vbucket 199 2016-04-07T03:50:19.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65520 Milliseconds Vbucket 199 2016-04-07T03:50:19.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65550 Milliseconds Vbucket 199 2016-04-07T03:50:19.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65580 Milliseconds Vbucket 199 2016-04-07T03:50:19.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65610 Milliseconds Vbucket 199 2016-04-07T03:50:20-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65640 Milliseconds Vbucket 199 2016-04-07T03:50:20.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65670 Milliseconds Vbucket 199 2016-04-07T03:50:20.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65700 Milliseconds Vbucket 199 2016-04-07T03:50:20.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65730 Milliseconds Vbucket 199 2016-04-07T03:50:20.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65760 Milliseconds Vbucket 199 2016-04-07T03:50:20.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65790 Milliseconds Vbucket 199 2016-04-07T03:50:20.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65820 Milliseconds Vbucket 199 2016-04-07T03:50:20.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65850 Milliseconds Vbucket 199 2016-04-07T03:50:20.237-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:20.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65880 Milliseconds Vbucket 199 2016-04-07T03:50:20.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65910 Milliseconds Vbucket 199 2016-04-07T03:50:20.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65940 Milliseconds Vbucket 199 2016-04-07T03:50:20.325-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14640000 QueuedCount 50000 2016-04-07T03:50:20.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 65970 Milliseconds Vbucket 199 2016-04-07T03:50:20.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66000 Milliseconds Vbucket 199 2016-04-07T03:50:20.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66030 Milliseconds Vbucket 199 2016-04-07T03:50:20.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66060 Milliseconds Vbucket 199 2016-04-07T03:50:20.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66090 Milliseconds Vbucket 199 2016-04-07T03:50:20.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66120 Milliseconds Vbucket 199 2016-04-07T03:50:20.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66150 Milliseconds Vbucket 199 2016-04-07T03:50:20.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66180 Milliseconds Vbucket 199 2016-04-07T03:50:20.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66210 Milliseconds Vbucket 199 2016-04-07T03:50:20.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66240 Milliseconds Vbucket 199 2016-04-07T03:50:20.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66270 Milliseconds Vbucket 199 2016-04-07T03:50:20.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66300 Milliseconds Vbucket 199 2016-04-07T03:50:20.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66330 Milliseconds Vbucket 199 2016-04-07T03:50:20.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66360 Milliseconds Vbucket 199 2016-04-07T03:50:20.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66390 Milliseconds Vbucket 199 2016-04-07T03:50:20.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66420 Milliseconds Vbucket 199 2016-04-07T03:50:20.808-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14650000 QueuedCount 50000 2016-04-07T03:50:20.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66450 Milliseconds Vbucket 199 2016-04-07T03:50:20.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66480 Milliseconds Vbucket 199 2016-04-07T03:50:20.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66510 Milliseconds Vbucket 199 2016-04-07T03:50:20.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66540 Milliseconds Vbucket 199 2016-04-07T03:50:20.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66570 Milliseconds Vbucket 199 2016-04-07T03:50:20.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66600 Milliseconds Vbucket 199 2016-04-07T03:50:20.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66630 Milliseconds Vbucket 199 2016-04-07T03:50:21.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66660 Milliseconds Vbucket 199 2016-04-07T03:50:21.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66690 Milliseconds Vbucket 199 2016-04-07T03:50:21.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66720 Milliseconds Vbucket 199 2016-04-07T03:50:21.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66750 Milliseconds Vbucket 199 2016-04-07T03:50:21.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66780 Milliseconds Vbucket 199 2016-04-07T03:50:21.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66810 Milliseconds Vbucket 199 2016-04-07T03:50:21.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66840 Milliseconds Vbucket 199 2016-04-07T03:50:21.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66870 Milliseconds Vbucket 199 2016-04-07T03:50:21.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66900 Milliseconds Vbucket 199 2016-04-07T03:50:21.274-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14660000 QueuedCount 50000 2016-04-07T03:50:21.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66930 Milliseconds Vbucket 199 2016-04-07T03:50:21.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66960 Milliseconds Vbucket 199 2016-04-07T03:50:21.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 66990 Milliseconds Vbucket 199 2016-04-07T03:50:21.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67020 Milliseconds Vbucket 199 2016-04-07T03:50:21.385-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:21.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67050 Milliseconds Vbucket 199 2016-04-07T03:50:21.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67080 Milliseconds Vbucket 199 2016-04-07T03:50:21.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67110 Milliseconds Vbucket 199 2016-04-07T03:50:21.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67140 Milliseconds Vbucket 199 2016-04-07T03:50:21.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67170 Milliseconds Vbucket 199 2016-04-07T03:50:21.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67200 Milliseconds Vbucket 199 2016-04-07T03:50:21.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67230 Milliseconds Vbucket 199 2016-04-07T03:50:21.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67260 Milliseconds Vbucket 199 2016-04-07T03:50:21.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67290 Milliseconds Vbucket 199 2016-04-07T03:50:21.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67320 Milliseconds Vbucket 199 2016-04-07T03:50:21.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67350 Milliseconds Vbucket 199 2016-04-07T03:50:21.733-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14670000 QueuedCount 50000 2016-04-07T03:50:21.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67380 Milliseconds Vbucket 199 2016-04-07T03:50:21.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67410 Milliseconds Vbucket 199 2016-04-07T03:50:21.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67440 Milliseconds Vbucket 199 2016-04-07T03:50:21.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67470 Milliseconds Vbucket 199 2016-04-07T03:50:21.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67500 Milliseconds Vbucket 199 2016-04-07T03:50:21.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67530 Milliseconds Vbucket 199 2016-04-07T03:50:21.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67560 Milliseconds Vbucket 199 2016-04-07T03:50:21.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67590 Milliseconds Vbucket 199 2016-04-07T03:50:21.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67620 Milliseconds Vbucket 199 2016-04-07T03:50:22.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67650 Milliseconds Vbucket 199 2016-04-07T03:50:22.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67680 Milliseconds Vbucket 199 2016-04-07T03:50:22.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67710 Milliseconds Vbucket 199 2016-04-07T03:50:22.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67740 Milliseconds Vbucket 199 2016-04-07T03:50:22.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67770 Milliseconds Vbucket 199 2016-04-07T03:50:22.134-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14680000 QueuedCount 50000 2016-04-07T03:50:22.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67800 Milliseconds Vbucket 199 2016-04-07T03:50:22.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67830 Milliseconds Vbucket 199 2016-04-07T03:50:22.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67860 Milliseconds Vbucket 199 2016-04-07T03:50:22.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67890 Milliseconds Vbucket 199 2016-04-07T03:50:22.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67920 Milliseconds Vbucket 199 2016-04-07T03:50:22.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67950 Milliseconds Vbucket 199 2016-04-07T03:50:22.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 67980 Milliseconds Vbucket 199 2016-04-07T03:50:22.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68010 Milliseconds Vbucket 199 2016-04-07T03:50:22.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68040 Milliseconds Vbucket 199 2016-04-07T03:50:22.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68070 Milliseconds Vbucket 199 2016-04-07T03:50:22.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68100 Milliseconds Vbucket 199 2016-04-07T03:50:22.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68130 Milliseconds Vbucket 199 2016-04-07T03:50:22.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68160 Milliseconds Vbucket 199 2016-04-07T03:50:22.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68190 Milliseconds Vbucket 199 2016-04-07T03:50:22.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68220 Milliseconds Vbucket 199 2016-04-07T03:50:22.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68250 Milliseconds Vbucket 199 2016-04-07T03:50:22.638-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14690000 QueuedCount 50000 2016-04-07T03:50:22.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68280 Milliseconds Vbucket 199 2016-04-07T03:50:22.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68310 Milliseconds Vbucket 199 2016-04-07T03:50:22.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68340 Milliseconds Vbucket 199 2016-04-07T03:50:22.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68370 Milliseconds Vbucket 199 2016-04-07T03:50:22.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68400 Milliseconds Vbucket 199 2016-04-07T03:50:22.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68430 Milliseconds Vbucket 199 2016-04-07T03:50:22.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68460 Milliseconds Vbucket 199 2016-04-07T03:50:22.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68490 Milliseconds Vbucket 199 2016-04-07T03:50:22.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68520 Milliseconds Vbucket 199 2016-04-07T03:50:22.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68550 Milliseconds Vbucket 199 2016-04-07T03:50:22.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68580 Milliseconds Vbucket 199 2016-04-07T03:50:22.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68610 Milliseconds Vbucket 199 2016-04-07T03:50:23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68640 Milliseconds Vbucket 199 2016-04-07T03:50:23.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68670 Milliseconds Vbucket 199 2016-04-07T03:50:23.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68700 Milliseconds Vbucket 199 2016-04-07T03:50:23.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68730 Milliseconds Vbucket 199 2016-04-07T03:50:23.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68760 Milliseconds Vbucket 199 2016-04-07T03:50:23.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68790 Milliseconds Vbucket 199 2016-04-07T03:50:23.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68820 Milliseconds Vbucket 199 2016-04-07T03:50:23.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68850 Milliseconds Vbucket 199 2016-04-07T03:50:23.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68880 Milliseconds Vbucket 199 2016-04-07T03:50:23.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68910 Milliseconds Vbucket 199 2016-04-07T03:50:23.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68940 Milliseconds Vbucket 199 2016-04-07T03:50:23.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 68970 Milliseconds Vbucket 199 2016-04-07T03:50:23.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69000 Milliseconds Vbucket 199 2016-04-07T03:50:23.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69030 Milliseconds Vbucket 199 2016-04-07T03:50:23.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69060 Milliseconds Vbucket 199 2016-04-07T03:50:23.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69090 Milliseconds Vbucket 199 2016-04-07T03:50:23.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69120 Milliseconds Vbucket 199 2016-04-07T03:50:23.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69150 Milliseconds Vbucket 199 2016-04-07T03:50:23.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69180 Milliseconds Vbucket 199 2016-04-07T03:50:23.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69210 Milliseconds Vbucket 199 2016-04-07T03:50:23.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69240 Milliseconds Vbucket 199 2016-04-07T03:50:23.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69270 Milliseconds Vbucket 199 2016-04-07T03:50:23.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69300 Milliseconds Vbucket 199 2016-04-07T03:50:23.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69330 Milliseconds Vbucket 199 2016-04-07T03:50:23.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69360 Milliseconds Vbucket 199 2016-04-07T03:50:23.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69390 Milliseconds Vbucket 199 2016-04-07T03:50:23.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69420 Milliseconds Vbucket 199 2016-04-07T03:50:23.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69450 Milliseconds Vbucket 199 2016-04-07T03:50:23.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69480 Milliseconds Vbucket 199 2016-04-07T03:50:23.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69510 Milliseconds Vbucket 199 2016-04-07T03:50:23.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69540 Milliseconds Vbucket 199 2016-04-07T03:50:23.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69570 Milliseconds Vbucket 199 2016-04-07T03:50:23.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69600 Milliseconds Vbucket 199 2016-04-07T03:50:23.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69630 Milliseconds Vbucket 199 2016-04-07T03:50:24.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69660 Milliseconds Vbucket 199 2016-04-07T03:50:24.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69690 Milliseconds Vbucket 199 2016-04-07T03:50:24.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69720 Milliseconds Vbucket 199 2016-04-07T03:50:24.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69750 Milliseconds Vbucket 199 2016-04-07T03:50:24.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69780 Milliseconds Vbucket 199 2016-04-07T03:50:24.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69810 Milliseconds Vbucket 199 2016-04-07T03:50:24.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69840 Milliseconds Vbucket 199 2016-04-07T03:50:24.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69870 Milliseconds Vbucket 199 2016-04-07T03:50:24.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69900 Milliseconds Vbucket 199 2016-04-07T03:50:24.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69930 Milliseconds Vbucket 199 2016-04-07T03:50:24.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69960 Milliseconds Vbucket 199 2016-04-07T03:50:24.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 69990 Milliseconds Vbucket 199 2016-04-07T03:50:24.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70020 Milliseconds Vbucket 199 2016-04-07T03:50:24.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70050 Milliseconds Vbucket 199 2016-04-07T03:50:24.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70080 Milliseconds Vbucket 199 2016-04-07T03:50:24.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70110 Milliseconds Vbucket 199 2016-04-07T03:50:24.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70140 Milliseconds Vbucket 199 2016-04-07T03:50:24.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70170 Milliseconds Vbucket 199 2016-04-07T03:50:24.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70200 Milliseconds Vbucket 199 2016-04-07T03:50:24.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70230 Milliseconds Vbucket 199 2016-04-07T03:50:24.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70260 Milliseconds Vbucket 199 2016-04-07T03:50:24.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70290 Milliseconds Vbucket 199 2016-04-07T03:50:24.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70320 Milliseconds Vbucket 199 2016-04-07T03:50:24.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70350 Milliseconds Vbucket 199 2016-04-07T03:50:24.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70380 Milliseconds Vbucket 199 2016-04-07T03:50:24.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70410 Milliseconds Vbucket 199 2016-04-07T03:50:24.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70440 Milliseconds Vbucket 199 2016-04-07T03:50:24.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70470 Milliseconds Vbucket 199 2016-04-07T03:50:24.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70500 Milliseconds Vbucket 199 2016-04-07T03:50:24.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70530 Milliseconds Vbucket 199 2016-04-07T03:50:24.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70560 Milliseconds Vbucket 199 2016-04-07T03:50:24.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70590 Milliseconds Vbucket 199 2016-04-07T03:50:24.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70620 Milliseconds Vbucket 199 2016-04-07T03:50:25.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70650 Milliseconds Vbucket 199 2016-04-07T03:50:25.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70680 Milliseconds Vbucket 199 2016-04-07T03:50:25.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70710 Milliseconds Vbucket 199 2016-04-07T03:50:25.084-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14700000 QueuedCount 44893 2016-04-07T03:50:25.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70740 Milliseconds Vbucket 199 2016-04-07T03:50:25.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70770 Milliseconds Vbucket 199 2016-04-07T03:50:25.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70800 Milliseconds Vbucket 199 2016-04-07T03:50:25.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70830 Milliseconds Vbucket 199 2016-04-07T03:50:25.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70860 Milliseconds Vbucket 199 2016-04-07T03:50:25.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70890 Milliseconds Vbucket 199 2016-04-07T03:50:25.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70920 Milliseconds Vbucket 199 2016-04-07T03:50:25.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70950 Milliseconds Vbucket 199 2016-04-07T03:50:25.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 70980 Milliseconds Vbucket 199 2016-04-07T03:50:25.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71010 Milliseconds Vbucket 199 2016-04-07T03:50:25.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71040 Milliseconds Vbucket 199 2016-04-07T03:50:25.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71070 Milliseconds Vbucket 199 2016-04-07T03:50:25.453-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14710000 QueuedCount 34893 2016-04-07T03:50:25.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71100 Milliseconds Vbucket 199 2016-04-07T03:50:25.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71130 Milliseconds Vbucket 199 2016-04-07T03:50:25.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71160 Milliseconds Vbucket 199 2016-04-07T03:50:25.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71190 Milliseconds Vbucket 199 2016-04-07T03:50:25.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71220 Milliseconds Vbucket 199 2016-04-07T03:50:25.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71250 Milliseconds Vbucket 199 2016-04-07T03:50:25.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71280 Milliseconds Vbucket 199 2016-04-07T03:50:25.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71310 Milliseconds Vbucket 199 2016-04-07T03:50:25.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71340 Milliseconds Vbucket 199 2016-04-07T03:50:25.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71370 Milliseconds Vbucket 199 2016-04-07T03:50:25.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71400 Milliseconds Vbucket 199 2016-04-07T03:50:25.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71430 Milliseconds Vbucket 199 2016-04-07T03:50:25.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71460 Milliseconds Vbucket 199 2016-04-07T03:50:25.838-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14720000 QueuedCount 24893 2016-04-07T03:50:25.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71490 Milliseconds Vbucket 199 2016-04-07T03:50:25.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71520 Milliseconds Vbucket 199 2016-04-07T03:50:25.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71550 Milliseconds Vbucket 199 2016-04-07T03:50:25.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71580 Milliseconds Vbucket 199 2016-04-07T03:50:25.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71610 Milliseconds Vbucket 199 2016-04-07T03:50:26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71640 Milliseconds Vbucket 199 2016-04-07T03:50:26.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71670 Milliseconds Vbucket 199 2016-04-07T03:50:26.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71700 Milliseconds Vbucket 199 2016-04-07T03:50:26.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71730 Milliseconds Vbucket 199 2016-04-07T03:50:26.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71760 Milliseconds Vbucket 199 2016-04-07T03:50:26.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71790 Milliseconds Vbucket 199 2016-04-07T03:50:26.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71820 Milliseconds Vbucket 199 2016-04-07T03:50:26.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71850 Milliseconds Vbucket 199 2016-04-07T03:50:26.221-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14730000 QueuedCount 14893 2016-04-07T03:50:26.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71880 Milliseconds Vbucket 199 2016-04-07T03:50:26.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71910 Milliseconds Vbucket 199 2016-04-07T03:50:26.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71940 Milliseconds Vbucket 199 2016-04-07T03:50:26.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 71970 Milliseconds Vbucket 199 2016-04-07T03:50:26.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72000 Milliseconds Vbucket 199 2016-04-07T03:50:26.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72030 Milliseconds Vbucket 199 2016-04-07T03:50:26.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72060 Milliseconds Vbucket 199 2016-04-07T03:50:26.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72090 Milliseconds Vbucket 199 2016-04-07T03:50:26.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72120 Milliseconds Vbucket 199 2016-04-07T03:50:26.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72150 Milliseconds Vbucket 199 2016-04-07T03:50:26.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72180 Milliseconds Vbucket 199 2016-04-07T03:50:26.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72210 Milliseconds Vbucket 199 2016-04-07T03:50:26.579-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14740000 QueuedCount 4893 2016-04-07T03:50:26.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72240 Milliseconds Vbucket 199 2016-04-07T03:50:26.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72270 Milliseconds Vbucket 199 2016-04-07T03:50:26.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72300 Milliseconds Vbucket 199 2016-04-07T03:50:26.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72330 Milliseconds Vbucket 199 2016-04-07T03:50:26.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72360 Milliseconds Vbucket 199 2016-04-07T03:50:26.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72390 Milliseconds Vbucket 199 2016-04-07T03:50:26.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72420 Milliseconds Vbucket 199 2016-04-07T03:50:26.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72450 Milliseconds Vbucket 199 2016-04-07T03:50:26.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72480 Milliseconds Vbucket 199 2016-04-07T03:50:26.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72510 Milliseconds Vbucket 199 2016-04-07T03:50:26.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72540 Milliseconds Vbucket 199 2016-04-07T03:50:26.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72570 Milliseconds Vbucket 199 2016-04-07T03:50:26.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72600 Milliseconds Vbucket 199 2016-04-07T03:50:26.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72630 Milliseconds Vbucket 199 2016-04-07T03:50:27.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72660 Milliseconds Vbucket 199 2016-04-07T03:50:27.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72690 Milliseconds Vbucket 199 2016-04-07T03:50:27.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72720 Milliseconds Vbucket 199 2016-04-07T03:50:27.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72750 Milliseconds Vbucket 199 2016-04-07T03:50:27.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72780 Milliseconds Vbucket 199 2016-04-07T03:50:27.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72810 Milliseconds Vbucket 199 2016-04-07T03:50:27.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72840 Milliseconds Vbucket 199 2016-04-07T03:50:27.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72870 Milliseconds Vbucket 199 2016-04-07T03:50:27.245-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.986µs CommitTime 496.1296ms TotalFlushTime 9m51.218756397s TotalCommitTime 40.190879479s 2016-04-07T03:50:27.245-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 14744893, 14744893, 110 committed:true 2016-04-07T03:50:27.245-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 14744893, 14744893, 110 committed:true) SnapCreateDur 505.894702ms SnapOpenDur 38.805µs 2016-04-07T03:50:27.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72900 Milliseconds Vbucket 199 2016-04-07T03:50:27.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72930 Milliseconds Vbucket 199 2016-04-07T03:50:27.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72960 Milliseconds Vbucket 199 2016-04-07T03:50:27.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 72990 Milliseconds Vbucket 199 2016-04-07T03:50:27.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73020 Milliseconds Vbucket 199 2016-04-07T03:50:27.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73050 Milliseconds Vbucket 199 2016-04-07T03:50:27.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73080 Milliseconds Vbucket 199 2016-04-07T03:50:27.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73110 Milliseconds Vbucket 199 2016-04-07T03:50:27.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73140 Milliseconds Vbucket 199 2016-04-07T03:50:27.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73170 Milliseconds Vbucket 199 2016-04-07T03:50:27.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73200 Milliseconds Vbucket 199 2016-04-07T03:50:27.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73230 Milliseconds Vbucket 199 2016-04-07T03:50:27.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73260 Milliseconds Vbucket 199 2016-04-07T03:50:27.625-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14750000 QueuedCount 27430 2016-04-07T03:50:27.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73290 Milliseconds Vbucket 199 2016-04-07T03:50:27.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73320 Milliseconds Vbucket 199 2016-04-07T03:50:27.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73350 Milliseconds Vbucket 199 2016-04-07T03:50:27.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73380 Milliseconds Vbucket 199 2016-04-07T03:50:27.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73410 Milliseconds Vbucket 199 2016-04-07T03:50:27.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73440 Milliseconds Vbucket 199 2016-04-07T03:50:27.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73470 Milliseconds Vbucket 199 2016-04-07T03:50:27.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73500 Milliseconds Vbucket 199 2016-04-07T03:50:27.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73530 Milliseconds Vbucket 199 2016-04-07T03:50:27.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73560 Milliseconds Vbucket 199 2016-04-07T03:50:27.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73590 Milliseconds Vbucket 199 2016-04-07T03:50:27.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73620 Milliseconds Vbucket 199 2016-04-07T03:50:28.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73650 Milliseconds Vbucket 199 2016-04-07T03:50:28.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73680 Milliseconds Vbucket 199 2016-04-07T03:50:28.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73710 Milliseconds Vbucket 199 2016-04-07T03:50:28.086-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14760000 QueuedCount 17430 2016-04-07T03:50:28.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73740 Milliseconds Vbucket 199 2016-04-07T03:50:28.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73770 Milliseconds Vbucket 199 2016-04-07T03:50:28.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73800 Milliseconds Vbucket 199 2016-04-07T03:50:28.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73830 Milliseconds Vbucket 199 2016-04-07T03:50:28.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73860 Milliseconds Vbucket 199 2016-04-07T03:50:28.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73890 Milliseconds Vbucket 199 2016-04-07T03:50:28.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73920 Milliseconds Vbucket 199 2016-04-07T03:50:28.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73950 Milliseconds Vbucket 199 2016-04-07T03:50:28.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 73980 Milliseconds Vbucket 199 2016-04-07T03:50:28.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74010 Milliseconds Vbucket 199 2016-04-07T03:50:28.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74040 Milliseconds Vbucket 199 2016-04-07T03:50:28.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74070 Milliseconds Vbucket 199 2016-04-07T03:50:28.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74100 Milliseconds Vbucket 199 2016-04-07T03:50:28.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74130 Milliseconds Vbucket 199 2016-04-07T03:50:28.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74160 Milliseconds Vbucket 199 2016-04-07T03:50:28.547-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14770000 QueuedCount 7430 2016-04-07T03:50:28.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74190 Milliseconds Vbucket 199 2016-04-07T03:50:28.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74220 Milliseconds Vbucket 199 2016-04-07T03:50:28.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74250 Milliseconds Vbucket 199 2016-04-07T03:50:28.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74280 Milliseconds Vbucket 199 2016-04-07T03:50:28.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74310 Milliseconds Vbucket 199 2016-04-07T03:50:28.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74340 Milliseconds Vbucket 199 2016-04-07T03:50:28.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74370 Milliseconds Vbucket 199 2016-04-07T03:50:28.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74400 Milliseconds Vbucket 199 2016-04-07T03:50:28.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74430 Milliseconds Vbucket 199 2016-04-07T03:50:28.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74460 Milliseconds Vbucket 199 2016-04-07T03:50:28.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74490 Milliseconds Vbucket 199 2016-04-07T03:50:28.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74520 Milliseconds Vbucket 199 2016-04-07T03:50:28.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74550 Milliseconds Vbucket 199 2016-04-07T03:50:28.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74580 Milliseconds Vbucket 199 2016-04-07T03:50:28.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74610 Milliseconds Vbucket 199 2016-04-07T03:50:29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74640 Milliseconds Vbucket 199 2016-04-07T03:50:29.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74670 Milliseconds Vbucket 199 2016-04-07T03:50:29.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74700 Milliseconds Vbucket 199 2016-04-07T03:50:29.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74730 Milliseconds Vbucket 199 2016-04-07T03:50:29.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74760 Milliseconds Vbucket 199 2016-04-07T03:50:29.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74790 Milliseconds Vbucket 199 2016-04-07T03:50:29.156-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.25µs CommitTime 332.632197ms TotalFlushTime 9m52.744227355s TotalCommitTime 40.523511676s 2016-04-07T03:50:29.156-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 14777430, 14777430, 111 committed:true 2016-04-07T03:50:29.156-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 14777430, 14777430, 111 committed:true) SnapCreateDur 345.094245ms SnapOpenDur 36.505µs 2016-04-07T03:50:29.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74820 Milliseconds Vbucket 199 2016-04-07T03:50:29.191-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:29.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74850 Milliseconds Vbucket 199 2016-04-07T03:50:29.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74880 Milliseconds Vbucket 199 2016-04-07T03:50:29.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74910 Milliseconds Vbucket 199 2016-04-07T03:50:29.288-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14780000 QueuedCount 50000 2016-04-07T03:50:29.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74940 Milliseconds Vbucket 199 2016-04-07T03:50:29.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 74970 Milliseconds Vbucket 199 2016-04-07T03:50:29.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75000 Milliseconds Vbucket 199 2016-04-07T03:50:29.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75030 Milliseconds Vbucket 199 2016-04-07T03:50:29.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75060 Milliseconds Vbucket 199 2016-04-07T03:50:29.431-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:29.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75090 Milliseconds Vbucket 199 2016-04-07T03:50:29.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75120 Milliseconds Vbucket 199 2016-04-07T03:50:29.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75150 Milliseconds Vbucket 199 2016-04-07T03:50:29.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75180 Milliseconds Vbucket 199 2016-04-07T03:50:29.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75210 Milliseconds Vbucket 199 2016-04-07T03:50:29.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75240 Milliseconds Vbucket 199 2016-04-07T03:50:29.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75270 Milliseconds Vbucket 199 2016-04-07T03:50:29.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75300 Milliseconds Vbucket 199 2016-04-07T03:50:29.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75330 Milliseconds Vbucket 199 2016-04-07T03:50:29.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75360 Milliseconds Vbucket 199 2016-04-07T03:50:29.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75390 Milliseconds Vbucket 199 2016-04-07T03:50:29.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75420 Milliseconds Vbucket 199 2016-04-07T03:50:29.798-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14790000 QueuedCount 50000 2016-04-07T03:50:29.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75450 Milliseconds Vbucket 199 2016-04-07T03:50:29.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75480 Milliseconds Vbucket 199 2016-04-07T03:50:29.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75510 Milliseconds Vbucket 199 2016-04-07T03:50:29.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75540 Milliseconds Vbucket 199 2016-04-07T03:50:29.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75570 Milliseconds Vbucket 199 2016-04-07T03:50:29.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75600 Milliseconds Vbucket 199 2016-04-07T03:50:29.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75630 Milliseconds Vbucket 199 2016-04-07T03:50:30.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75660 Milliseconds Vbucket 199 2016-04-07T03:50:30.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75690 Milliseconds Vbucket 199 2016-04-07T03:50:30.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75720 Milliseconds Vbucket 199 2016-04-07T03:50:30.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75750 Milliseconds Vbucket 199 2016-04-07T03:50:30.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75780 Milliseconds Vbucket 199 2016-04-07T03:50:30.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75810 Milliseconds Vbucket 199 2016-04-07T03:50:30.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75840 Milliseconds Vbucket 199 2016-04-07T03:50:30.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75870 Milliseconds Vbucket 199 2016-04-07T03:50:30.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75900 Milliseconds Vbucket 199 2016-04-07T03:50:30.276-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14800000 QueuedCount 50000 2016-04-07T03:50:30.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75930 Milliseconds Vbucket 199 2016-04-07T03:50:30.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75960 Milliseconds Vbucket 199 2016-04-07T03:50:30.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 75990 Milliseconds Vbucket 199 2016-04-07T03:50:30.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76020 Milliseconds Vbucket 199 2016-04-07T03:50:30.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76050 Milliseconds Vbucket 199 2016-04-07T03:50:30.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76080 Milliseconds Vbucket 199 2016-04-07T03:50:30.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76110 Milliseconds Vbucket 199 2016-04-07T03:50:30.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76140 Milliseconds Vbucket 199 2016-04-07T03:50:30.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76170 Milliseconds Vbucket 199 2016-04-07T03:50:30.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76200 Milliseconds Vbucket 199 2016-04-07T03:50:30.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76230 Milliseconds Vbucket 199 2016-04-07T03:50:30.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76260 Milliseconds Vbucket 199 2016-04-07T03:50:30.646-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:30.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76290 Milliseconds Vbucket 199 2016-04-07T03:50:30.666-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14810000 QueuedCount 48184 2016-04-07T03:50:30.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76320 Milliseconds Vbucket 199 2016-04-07T03:50:30.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76350 Milliseconds Vbucket 199 2016-04-07T03:50:30.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76380 Milliseconds Vbucket 199 2016-04-07T03:50:30.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76410 Milliseconds Vbucket 199 2016-04-07T03:50:30.989-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76440 Milliseconds Vbucket 199 2016-04-07T03:50:31.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76470 Milliseconds Vbucket 199 2016-04-07T03:50:31.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76500 Milliseconds Vbucket 199 2016-04-07T03:50:31.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76530 Milliseconds Vbucket 199 2016-04-07T03:50:31.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76560 Milliseconds Vbucket 199 2016-04-07T03:50:31.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76590 Milliseconds Vbucket 199 2016-04-07T03:50:31.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76620 Milliseconds Vbucket 199 2016-04-07T03:50:31.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76650 Milliseconds Vbucket 199 2016-04-07T03:50:31.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76680 Milliseconds Vbucket 199 2016-04-07T03:50:31.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76710 Milliseconds Vbucket 199 2016-04-07T03:50:31.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76740 Milliseconds Vbucket 199 2016-04-07T03:50:31.293-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14820000 QueuedCount 50000 2016-04-07T03:50:31.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76770 Milliseconds Vbucket 199 2016-04-07T03:50:31.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76800 Milliseconds Vbucket 199 2016-04-07T03:50:31.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76830 Milliseconds Vbucket 199 2016-04-07T03:50:31.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76860 Milliseconds Vbucket 199 2016-04-07T03:50:31.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76890 Milliseconds Vbucket 199 2016-04-07T03:50:31.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76920 Milliseconds Vbucket 199 2016-04-07T03:50:31.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76950 Milliseconds Vbucket 199 2016-04-07T03:50:31.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 76980 Milliseconds Vbucket 199 2016-04-07T03:50:31.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77010 Milliseconds Vbucket 199 2016-04-07T03:50:31.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77040 Milliseconds Vbucket 199 2016-04-07T03:50:31.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77070 Milliseconds Vbucket 199 2016-04-07T03:50:31.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77100 Milliseconds Vbucket 199 2016-04-07T03:50:31.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77130 Milliseconds Vbucket 199 2016-04-07T03:50:31.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77160 Milliseconds Vbucket 199 2016-04-07T03:50:31.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77190 Milliseconds Vbucket 199 2016-04-07T03:50:31.748-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14830000 QueuedCount 50000 2016-04-07T03:50:31.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77220 Milliseconds Vbucket 199 2016-04-07T03:50:31.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77250 Milliseconds Vbucket 199 2016-04-07T03:50:31.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77280 Milliseconds Vbucket 199 2016-04-07T03:50:31.836-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:31.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77310 Milliseconds Vbucket 199 2016-04-07T03:50:31.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77340 Milliseconds Vbucket 199 2016-04-07T03:50:31.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77370 Milliseconds Vbucket 199 2016-04-07T03:50:31.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77400 Milliseconds Vbucket 199 2016-04-07T03:50:31.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77430 Milliseconds Vbucket 199 2016-04-07T03:50:32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77460 Milliseconds Vbucket 199 2016-04-07T03:50:32.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77490 Milliseconds Vbucket 199 2016-04-07T03:50:32.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77520 Milliseconds Vbucket 199 2016-04-07T03:50:32.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77550 Milliseconds Vbucket 199 2016-04-07T03:50:32.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77580 Milliseconds Vbucket 199 2016-04-07T03:50:32.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77610 Milliseconds Vbucket 199 2016-04-07T03:50:32.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77640 Milliseconds Vbucket 199 2016-04-07T03:50:32.194-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14840000 QueuedCount 50000 2016-04-07T03:50:32.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77670 Milliseconds Vbucket 199 2016-04-07T03:50:32.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77700 Milliseconds Vbucket 199 2016-04-07T03:50:32.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77730 Milliseconds Vbucket 199 2016-04-07T03:50:32.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77760 Milliseconds Vbucket 199 2016-04-07T03:50:32.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77790 Milliseconds Vbucket 199 2016-04-07T03:50:32.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77820 Milliseconds Vbucket 199 2016-04-07T03:50:32.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77850 Milliseconds Vbucket 199 2016-04-07T03:50:32.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77880 Milliseconds Vbucket 199 2016-04-07T03:50:32.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77910 Milliseconds Vbucket 199 2016-04-07T03:50:32.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77940 Milliseconds Vbucket 199 2016-04-07T03:50:32.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 77970 Milliseconds Vbucket 199 2016-04-07T03:50:32.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78000 Milliseconds Vbucket 199 2016-04-07T03:50:32.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78030 Milliseconds Vbucket 199 2016-04-07T03:50:32.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78060 Milliseconds Vbucket 199 2016-04-07T03:50:32.625-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14850000 QueuedCount 50000 2016-04-07T03:50:32.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78090 Milliseconds Vbucket 199 2016-04-07T03:50:32.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78120 Milliseconds Vbucket 199 2016-04-07T03:50:32.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78150 Milliseconds Vbucket 199 2016-04-07T03:50:32.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78180 Milliseconds Vbucket 199 2016-04-07T03:50:32.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78210 Milliseconds Vbucket 199 2016-04-07T03:50:32.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78240 Milliseconds Vbucket 199 2016-04-07T03:50:32.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78270 Milliseconds Vbucket 199 2016-04-07T03:50:32.831-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:32.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78300 Milliseconds Vbucket 199 2016-04-07T03:50:32.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78330 Milliseconds Vbucket 199 2016-04-07T03:50:32.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78360 Milliseconds Vbucket 199 2016-04-07T03:50:32.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78390 Milliseconds Vbucket 199 2016-04-07T03:50:32.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78420 Milliseconds Vbucket 199 2016-04-07T03:50:32.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78450 Milliseconds Vbucket 199 2016-04-07T03:50:33.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78480 Milliseconds Vbucket 199 2016-04-07T03:50:33.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78510 Milliseconds Vbucket 199 2016-04-07T03:50:33.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78540 Milliseconds Vbucket 199 2016-04-07T03:50:33.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78570 Milliseconds Vbucket 199 2016-04-07T03:50:33.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78600 Milliseconds Vbucket 199 2016-04-07T03:50:33.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78630 Milliseconds Vbucket 199 2016-04-07T03:50:33.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78660 Milliseconds Vbucket 199 2016-04-07T03:50:33.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78690 Milliseconds Vbucket 199 2016-04-07T03:50:33.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78720 Milliseconds Vbucket 199 2016-04-07T03:50:33.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78750 Milliseconds Vbucket 199 2016-04-07T03:50:33.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78780 Milliseconds Vbucket 199 2016-04-07T03:50:33.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78810 Milliseconds Vbucket 199 2016-04-07T03:50:33.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78840 Milliseconds Vbucket 199 2016-04-07T03:50:33.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78870 Milliseconds Vbucket 199 2016-04-07T03:50:33.437-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14860000 QueuedCount 50000 2016-04-07T03:50:33.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78900 Milliseconds Vbucket 199 2016-04-07T03:50:33.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78930 Milliseconds Vbucket 199 2016-04-07T03:50:33.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78960 Milliseconds Vbucket 199 2016-04-07T03:50:33.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 78990 Milliseconds Vbucket 199 2016-04-07T03:50:33.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79020 Milliseconds Vbucket 199 2016-04-07T03:50:33.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79050 Milliseconds Vbucket 199 2016-04-07T03:50:33.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79080 Milliseconds Vbucket 199 2016-04-07T03:50:33.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79110 Milliseconds Vbucket 199 2016-04-07T03:50:33.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79140 Milliseconds Vbucket 199 2016-04-07T03:50:33.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79170 Milliseconds Vbucket 199 2016-04-07T03:50:33.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79200 Milliseconds Vbucket 199 2016-04-07T03:50:33.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79230 Milliseconds Vbucket 199 2016-04-07T03:50:33.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79260 Milliseconds Vbucket 199 2016-04-07T03:50:33.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79290 Milliseconds Vbucket 199 2016-04-07T03:50:33.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79320 Milliseconds Vbucket 199 2016-04-07T03:50:33.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79350 Milliseconds Vbucket 199 2016-04-07T03:50:33.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79380 Milliseconds Vbucket 199 2016-04-07T03:50:33.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79410 Milliseconds Vbucket 199 2016-04-07T03:50:33.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79440 Milliseconds Vbucket 199 2016-04-07T03:50:34.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79470 Milliseconds Vbucket 199 2016-04-07T03:50:34.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79500 Milliseconds Vbucket 199 2016-04-07T03:50:34.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79530 Milliseconds Vbucket 199 2016-04-07T03:50:34.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79560 Milliseconds Vbucket 199 2016-04-07T03:50:34.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79590 Milliseconds Vbucket 199 2016-04-07T03:50:34.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79620 Milliseconds Vbucket 199 2016-04-07T03:50:34.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79650 Milliseconds Vbucket 199 2016-04-07T03:50:34.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79680 Milliseconds Vbucket 199 2016-04-07T03:50:34.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79710 Milliseconds Vbucket 199 2016-04-07T03:50:34.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79740 Milliseconds Vbucket 199 2016-04-07T03:50:34.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79770 Milliseconds Vbucket 199 2016-04-07T03:50:34.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79800 Milliseconds Vbucket 199 2016-04-07T03:50:34.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79830 Milliseconds Vbucket 199 2016-04-07T03:50:34.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79860 Milliseconds Vbucket 199 2016-04-07T03:50:34.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79890 Milliseconds Vbucket 199 2016-04-07T03:50:34.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79920 Milliseconds Vbucket 199 2016-04-07T03:50:34.487-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14870000 QueuedCount 50000 2016-04-07T03:50:34.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79950 Milliseconds Vbucket 199 2016-04-07T03:50:34.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 79980 Milliseconds Vbucket 199 2016-04-07T03:50:34.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80010 Milliseconds Vbucket 199 2016-04-07T03:50:34.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80040 Milliseconds Vbucket 199 2016-04-07T03:50:34.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80070 Milliseconds Vbucket 199 2016-04-07T03:50:34.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80100 Milliseconds Vbucket 199 2016-04-07T03:50:34.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80130 Milliseconds Vbucket 199 2016-04-07T03:50:34.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80160 Milliseconds Vbucket 199 2016-04-07T03:50:34.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80190 Milliseconds Vbucket 199 2016-04-07T03:50:34.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80220 Milliseconds Vbucket 199 2016-04-07T03:50:34.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80250 Milliseconds Vbucket 199 2016-04-07T03:50:34.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80280 Milliseconds Vbucket 199 2016-04-07T03:50:34.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80310 Milliseconds Vbucket 199 2016-04-07T03:50:34.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80340 Milliseconds Vbucket 199 2016-04-07T03:50:34.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80370 Milliseconds Vbucket 199 2016-04-07T03:50:34.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80400 Milliseconds Vbucket 199 2016-04-07T03:50:34.943-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14880000 QueuedCount 48421 2016-04-07T03:50:34.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80430 Milliseconds Vbucket 199 2016-04-07T03:50:35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80460 Milliseconds Vbucket 199 2016-04-07T03:50:35.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80490 Milliseconds Vbucket 199 2016-04-07T03:50:35.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80520 Milliseconds Vbucket 199 2016-04-07T03:50:35.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80550 Milliseconds Vbucket 199 2016-04-07T03:50:35.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80580 Milliseconds Vbucket 199 2016-04-07T03:50:35.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80610 Milliseconds Vbucket 199 2016-04-07T03:50:35.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80640 Milliseconds Vbucket 199 2016-04-07T03:50:35.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80670 Milliseconds Vbucket 199 2016-04-07T03:50:35.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80700 Milliseconds Vbucket 199 2016-04-07T03:50:35.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80730 Milliseconds Vbucket 199 2016-04-07T03:50:35.293-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14890000 QueuedCount 38421 2016-04-07T03:50:35.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80760 Milliseconds Vbucket 199 2016-04-07T03:50:35.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80790 Milliseconds Vbucket 199 2016-04-07T03:50:35.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80820 Milliseconds Vbucket 199 2016-04-07T03:50:35.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80850 Milliseconds Vbucket 199 2016-04-07T03:50:35.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80880 Milliseconds Vbucket 199 2016-04-07T03:50:35.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80910 Milliseconds Vbucket 199 2016-04-07T03:50:35.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80940 Milliseconds Vbucket 199 2016-04-07T03:50:35.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 80970 Milliseconds Vbucket 199 2016-04-07T03:50:35.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81000 Milliseconds Vbucket 199 2016-04-07T03:50:35.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81030 Milliseconds Vbucket 199 2016-04-07T03:50:35.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81060 Milliseconds Vbucket 199 2016-04-07T03:50:35.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81090 Milliseconds Vbucket 199 2016-04-07T03:50:35.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81120 Milliseconds Vbucket 199 2016-04-07T03:50:35.679-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14900000 QueuedCount 28421 2016-04-07T03:50:35.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81150 Milliseconds Vbucket 199 2016-04-07T03:50:35.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81180 Milliseconds Vbucket 199 2016-04-07T03:50:35.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81210 Milliseconds Vbucket 199 2016-04-07T03:50:35.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81240 Milliseconds Vbucket 199 2016-04-07T03:50:35.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81270 Milliseconds Vbucket 199 2016-04-07T03:50:35.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81300 Milliseconds Vbucket 199 2016-04-07T03:50:35.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81330 Milliseconds Vbucket 199 2016-04-07T03:50:35.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81360 Milliseconds Vbucket 199 2016-04-07T03:50:35.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81390 Milliseconds Vbucket 199 2016-04-07T03:50:35.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81420 Milliseconds Vbucket 199 2016-04-07T03:50:35.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81450 Milliseconds Vbucket 199 2016-04-07T03:50:36.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81480 Milliseconds Vbucket 199 2016-04-07T03:50:36.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81510 Milliseconds Vbucket 199 2016-04-07T03:50:36.059-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14910000 QueuedCount 18421 2016-04-07T03:50:36.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81540 Milliseconds Vbucket 199 2016-04-07T03:50:36.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81570 Milliseconds Vbucket 199 2016-04-07T03:50:36.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81600 Milliseconds Vbucket 199 2016-04-07T03:50:36.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81630 Milliseconds Vbucket 199 2016-04-07T03:50:36.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81660 Milliseconds Vbucket 199 2016-04-07T03:50:36.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81690 Milliseconds Vbucket 199 2016-04-07T03:50:36.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81720 Milliseconds Vbucket 199 2016-04-07T03:50:36.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81750 Milliseconds Vbucket 199 2016-04-07T03:50:36.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81780 Milliseconds Vbucket 199 2016-04-07T03:50:36.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81810 Milliseconds Vbucket 199 2016-04-07T03:50:36.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81840 Milliseconds Vbucket 199 2016-04-07T03:50:36.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81870 Milliseconds Vbucket 199 2016-04-07T03:50:36.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81900 Milliseconds Vbucket 199 2016-04-07T03:50:36.449-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14920000 QueuedCount 8421 2016-04-07T03:50:36.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81930 Milliseconds Vbucket 199 2016-04-07T03:50:36.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81960 Milliseconds Vbucket 199 2016-04-07T03:50:36.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81990 Milliseconds Vbucket 199 2016-04-07T03:50:36.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82020 Milliseconds Vbucket 199 2016-04-07T03:50:36.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82050 Milliseconds Vbucket 199 2016-04-07T03:50:36.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82080 Milliseconds Vbucket 199 2016-04-07T03:50:36.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82110 Milliseconds Vbucket 199 2016-04-07T03:50:36.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82140 Milliseconds Vbucket 199 2016-04-07T03:50:36.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82170 Milliseconds Vbucket 199 2016-04-07T03:50:36.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82200 Milliseconds Vbucket 199 2016-04-07T03:50:36.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82230 Milliseconds Vbucket 199 2016-04-07T03:50:36.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82260 Milliseconds Vbucket 199 2016-04-07T03:50:36.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82290 Milliseconds Vbucket 199 2016-04-07T03:50:36.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82320 Milliseconds Vbucket 199 2016-04-07T03:50:36.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82350 Milliseconds Vbucket 199 2016-04-07T03:50:36.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82380 Milliseconds Vbucket 199 2016-04-07T03:50:36.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82410 Milliseconds Vbucket 199 2016-04-07T03:50:36.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82440 Milliseconds Vbucket 199 2016-04-07T03:50:37.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82470 Milliseconds Vbucket 199 2016-04-07T03:50:37.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82500 Milliseconds Vbucket 199 2016-04-07T03:50:37.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82530 Milliseconds Vbucket 199 2016-04-07T03:50:37.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82560 Milliseconds Vbucket 199 2016-04-07T03:50:37.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82590 Milliseconds Vbucket 199 2016-04-07T03:50:37.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82620 Milliseconds Vbucket 199 2016-04-07T03:50:37.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82650 Milliseconds Vbucket 199 2016-04-07T03:50:37.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82680 Milliseconds Vbucket 199 2016-04-07T03:50:37.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82710 Milliseconds Vbucket 199 2016-04-07T03:50:37.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82740 Milliseconds Vbucket 199 2016-04-07T03:50:37.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82770 Milliseconds Vbucket 199 2016-04-07T03:50:37.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82800 Milliseconds Vbucket 199 2016-04-07T03:50:37.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82830 Milliseconds Vbucket 199 2016-04-07T03:50:37.396-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.677µs CommitTime 596.455779ms TotalFlushTime 10m0.135653808s TotalCommitTime 41.119967455s 2016-04-07T03:50:37.396-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 14928421, 14928421, 112 committed:true 2016-04-07T03:50:37.396-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 14928421, 14928421, 112 committed:true) SnapCreateDur 607.375226ms SnapOpenDur 51.75µs 2016-04-07T03:50:37.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82860 Milliseconds Vbucket 199 2016-04-07T03:50:37.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82890 Milliseconds Vbucket 199 2016-04-07T03:50:37.431-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:37.441-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14930000 QueuedCount 23308 2016-04-07T03:50:37.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82920 Milliseconds Vbucket 199 2016-04-07T03:50:37.465-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:37.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82950 Milliseconds Vbucket 199 2016-04-07T03:50:37.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 82980 Milliseconds Vbucket 199 2016-04-07T03:50:37.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83010 Milliseconds Vbucket 199 2016-04-07T03:50:37.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83040 Milliseconds Vbucket 199 2016-04-07T03:50:37.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83070 Milliseconds Vbucket 199 2016-04-07T03:50:37.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83100 Milliseconds Vbucket 199 2016-04-07T03:50:37.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83130 Milliseconds Vbucket 199 2016-04-07T03:50:37.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83160 Milliseconds Vbucket 199 2016-04-07T03:50:37.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83190 Milliseconds Vbucket 199 2016-04-07T03:50:37.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83220 Milliseconds Vbucket 199 2016-04-07T03:50:37.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83250 Milliseconds Vbucket 199 2016-04-07T03:50:37.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83280 Milliseconds Vbucket 199 2016-04-07T03:50:37.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83310 Milliseconds Vbucket 199 2016-04-07T03:50:37.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83340 Milliseconds Vbucket 199 2016-04-07T03:50:37.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83370 Milliseconds Vbucket 199 2016-04-07T03:50:37.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83400 Milliseconds Vbucket 199 2016-04-07T03:50:37.951-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14940000 QueuedCount 50000 2016-04-07T03:50:37.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83430 Milliseconds Vbucket 199 2016-04-07T03:50:38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83460 Milliseconds Vbucket 199 2016-04-07T03:50:38.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83490 Milliseconds Vbucket 199 2016-04-07T03:50:38.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83520 Milliseconds Vbucket 199 2016-04-07T03:50:38.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83550 Milliseconds Vbucket 199 2016-04-07T03:50:38.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83580 Milliseconds Vbucket 199 2016-04-07T03:50:38.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83610 Milliseconds Vbucket 199 2016-04-07T03:50:38.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83640 Milliseconds Vbucket 199 2016-04-07T03:50:38.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83670 Milliseconds Vbucket 199 2016-04-07T03:50:38.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83700 Milliseconds Vbucket 199 2016-04-07T03:50:38.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83730 Milliseconds Vbucket 199 2016-04-07T03:50:38.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83760 Milliseconds Vbucket 199 2016-04-07T03:50:38.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83790 Milliseconds Vbucket 199 2016-04-07T03:50:38.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83820 Milliseconds Vbucket 199 2016-04-07T03:50:38.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83850 Milliseconds Vbucket 199 2016-04-07T03:50:38.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83880 Milliseconds Vbucket 199 2016-04-07T03:50:38.424-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14950000 QueuedCount 50000 2016-04-07T03:50:38.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83910 Milliseconds Vbucket 199 2016-04-07T03:50:38.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83940 Milliseconds Vbucket 199 2016-04-07T03:50:38.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 83970 Milliseconds Vbucket 199 2016-04-07T03:50:38.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84000 Milliseconds Vbucket 199 2016-04-07T03:50:38.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84030 Milliseconds Vbucket 199 2016-04-07T03:50:38.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84060 Milliseconds Vbucket 199 2016-04-07T03:50:38.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84090 Milliseconds Vbucket 199 2016-04-07T03:50:38.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84120 Milliseconds Vbucket 199 2016-04-07T03:50:38.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84150 Milliseconds Vbucket 199 2016-04-07T03:50:38.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84180 Milliseconds Vbucket 199 2016-04-07T03:50:38.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84210 Milliseconds Vbucket 199 2016-04-07T03:50:38.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84240 Milliseconds Vbucket 199 2016-04-07T03:50:38.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84270 Milliseconds Vbucket 199 2016-04-07T03:50:38.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84300 Milliseconds Vbucket 199 2016-04-07T03:50:38.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84330 Milliseconds Vbucket 199 2016-04-07T03:50:38.895-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14960000 QueuedCount 50000 2016-04-07T03:50:38.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84360 Milliseconds Vbucket 199 2016-04-07T03:50:38.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84390 Milliseconds Vbucket 199 2016-04-07T03:50:38.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84420 Milliseconds Vbucket 199 2016-04-07T03:50:38.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84450 Milliseconds Vbucket 199 2016-04-07T03:50:39.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84480 Milliseconds Vbucket 199 2016-04-07T03:50:39.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84510 Milliseconds Vbucket 199 2016-04-07T03:50:39.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84540 Milliseconds Vbucket 199 2016-04-07T03:50:39.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84570 Milliseconds Vbucket 199 2016-04-07T03:50:39.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84600 Milliseconds Vbucket 199 2016-04-07T03:50:39.158-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:39.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84630 Milliseconds Vbucket 199 2016-04-07T03:50:39.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84660 Milliseconds Vbucket 199 2016-04-07T03:50:39.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84690 Milliseconds Vbucket 199 2016-04-07T03:50:39.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84720 Milliseconds Vbucket 199 2016-04-07T03:50:39.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84750 Milliseconds Vbucket 199 2016-04-07T03:50:39.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84780 Milliseconds Vbucket 199 2016-04-07T03:50:39.344-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14970000 QueuedCount 50000 2016-04-07T03:50:39.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84810 Milliseconds Vbucket 199 2016-04-07T03:50:39.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84840 Milliseconds Vbucket 199 2016-04-07T03:50:39.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84870 Milliseconds Vbucket 199 2016-04-07T03:50:39.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84900 Milliseconds Vbucket 199 2016-04-07T03:50:39.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84930 Milliseconds Vbucket 199 2016-04-07T03:50:39.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84960 Milliseconds Vbucket 199 2016-04-07T03:50:39.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 84990 Milliseconds Vbucket 199 2016-04-07T03:50:39.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85020 Milliseconds Vbucket 199 2016-04-07T03:50:39.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85050 Milliseconds Vbucket 199 2016-04-07T03:50:39.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85080 Milliseconds Vbucket 199 2016-04-07T03:50:39.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85110 Milliseconds Vbucket 199 2016-04-07T03:50:39.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85140 Milliseconds Vbucket 199 2016-04-07T03:50:39.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85170 Milliseconds Vbucket 199 2016-04-07T03:50:39.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85200 Milliseconds Vbucket 199 2016-04-07T03:50:39.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85230 Milliseconds Vbucket 199 2016-04-07T03:50:39.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85260 Milliseconds Vbucket 199 2016-04-07T03:50:39.802-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14980000 QueuedCount 50000 2016-04-07T03:50:39.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85290 Milliseconds Vbucket 199 2016-04-07T03:50:39.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85320 Milliseconds Vbucket 199 2016-04-07T03:50:39.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85350 Milliseconds Vbucket 199 2016-04-07T03:50:39.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85380 Milliseconds Vbucket 199 2016-04-07T03:50:39.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85410 Milliseconds Vbucket 199 2016-04-07T03:50:39.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85440 Milliseconds Vbucket 199 2016-04-07T03:50:40.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85470 Milliseconds Vbucket 199 2016-04-07T03:50:40.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85500 Milliseconds Vbucket 199 2016-04-07T03:50:40.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85530 Milliseconds Vbucket 199 2016-04-07T03:50:40.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85560 Milliseconds Vbucket 199 2016-04-07T03:50:40.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85590 Milliseconds Vbucket 199 2016-04-07T03:50:40.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85620 Milliseconds Vbucket 199 2016-04-07T03:50:40.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85650 Milliseconds Vbucket 199 2016-04-07T03:50:40.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85680 Milliseconds Vbucket 199 2016-04-07T03:50:40.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85710 Milliseconds Vbucket 199 2016-04-07T03:50:40.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85740 Milliseconds Vbucket 199 2016-04-07T03:50:40.281-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 14990000 QueuedCount 50000 2016-04-07T03:50:40.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85770 Milliseconds Vbucket 199 2016-04-07T03:50:40.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85800 Milliseconds Vbucket 199 2016-04-07T03:50:40.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85830 Milliseconds Vbucket 199 2016-04-07T03:50:40.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85860 Milliseconds Vbucket 199 2016-04-07T03:50:40.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85890 Milliseconds Vbucket 199 2016-04-07T03:50:40.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85920 Milliseconds Vbucket 199 2016-04-07T03:50:40.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85950 Milliseconds Vbucket 199 2016-04-07T03:50:40.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 85980 Milliseconds Vbucket 199 2016-04-07T03:50:40.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86010 Milliseconds Vbucket 199 2016-04-07T03:50:40.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86040 Milliseconds Vbucket 199 2016-04-07T03:50:40.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86070 Milliseconds Vbucket 199 2016-04-07T03:50:40.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86100 Milliseconds Vbucket 199 2016-04-07T03:50:40.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86130 Milliseconds Vbucket 199 2016-04-07T03:50:40.675-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15000000 QueuedCount 49607 2016-04-07T03:50:40.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86160 Milliseconds Vbucket 199 2016-04-07T03:50:40.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86190 Milliseconds Vbucket 199 2016-04-07T03:50:40.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86220 Milliseconds Vbucket 199 2016-04-07T03:50:40.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86250 Milliseconds Vbucket 199 2016-04-07T03:50:40.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86280 Milliseconds Vbucket 199 2016-04-07T03:50:40.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86310 Milliseconds Vbucket 199 2016-04-07T03:50:40.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86340 Milliseconds Vbucket 199 2016-04-07T03:50:40.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86370 Milliseconds Vbucket 199 2016-04-07T03:50:40.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86400 Milliseconds Vbucket 199 2016-04-07T03:50:40.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86430 Milliseconds Vbucket 199 2016-04-07T03:50:41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86460 Milliseconds Vbucket 199 2016-04-07T03:50:41.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86490 Milliseconds Vbucket 199 2016-04-07T03:50:41.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86520 Milliseconds Vbucket 199 2016-04-07T03:50:41.087-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15010000 QueuedCount 39607 2016-04-07T03:50:41.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86550 Milliseconds Vbucket 199 2016-04-07T03:50:41.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86580 Milliseconds Vbucket 199 2016-04-07T03:50:41.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86610 Milliseconds Vbucket 199 2016-04-07T03:50:41.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86640 Milliseconds Vbucket 199 2016-04-07T03:50:41.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86670 Milliseconds Vbucket 199 2016-04-07T03:50:41.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86700 Milliseconds Vbucket 199 2016-04-07T03:50:41.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86730 Milliseconds Vbucket 199 2016-04-07T03:50:41.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86760 Milliseconds Vbucket 199 2016-04-07T03:50:41.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86790 Milliseconds Vbucket 199 2016-04-07T03:50:41.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86820 Milliseconds Vbucket 199 2016-04-07T03:50:41.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86850 Milliseconds Vbucket 199 2016-04-07T03:50:41.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86880 Milliseconds Vbucket 199 2016-04-07T03:50:41.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86910 Milliseconds Vbucket 199 2016-04-07T03:50:41.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86940 Milliseconds Vbucket 199 2016-04-07T03:50:41.48-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15020000 QueuedCount 29607 2016-04-07T03:50:41.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 86970 Milliseconds Vbucket 199 2016-04-07T03:50:41.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87000 Milliseconds Vbucket 199 2016-04-07T03:50:41.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87030 Milliseconds Vbucket 199 2016-04-07T03:50:41.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87060 Milliseconds Vbucket 199 2016-04-07T03:50:41.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87090 Milliseconds Vbucket 199 2016-04-07T03:50:41.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87120 Milliseconds Vbucket 199 2016-04-07T03:50:41.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87150 Milliseconds Vbucket 199 2016-04-07T03:50:41.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87180 Milliseconds Vbucket 199 2016-04-07T03:50:41.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87210 Milliseconds Vbucket 199 2016-04-07T03:50:41.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87240 Milliseconds Vbucket 199 2016-04-07T03:50:41.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87270 Milliseconds Vbucket 199 2016-04-07T03:50:41.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87300 Milliseconds Vbucket 199 2016-04-07T03:50:41.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87330 Milliseconds Vbucket 199 2016-04-07T03:50:41.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87360 Milliseconds Vbucket 199 2016-04-07T03:50:41.92-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15030000 QueuedCount 19607 2016-04-07T03:50:41.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87390 Milliseconds Vbucket 199 2016-04-07T03:50:41.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87420 Milliseconds Vbucket 199 2016-04-07T03:50:41.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87450 Milliseconds Vbucket 199 2016-04-07T03:50:42.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87480 Milliseconds Vbucket 199 2016-04-07T03:50:42.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87510 Milliseconds Vbucket 199 2016-04-07T03:50:42.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87540 Milliseconds Vbucket 199 2016-04-07T03:50:42.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87570 Milliseconds Vbucket 199 2016-04-07T03:50:42.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87600 Milliseconds Vbucket 199 2016-04-07T03:50:42.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87630 Milliseconds Vbucket 199 2016-04-07T03:50:42.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87660 Milliseconds Vbucket 199 2016-04-07T03:50:42.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87690 Milliseconds Vbucket 199 2016-04-07T03:50:42.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87720 Milliseconds Vbucket 199 2016-04-07T03:50:42.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87750 Milliseconds Vbucket 199 2016-04-07T03:50:42.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87780 Milliseconds Vbucket 199 2016-04-07T03:50:42.341-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15040000 QueuedCount 9607 2016-04-07T03:50:42.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87810 Milliseconds Vbucket 199 2016-04-07T03:50:42.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87840 Milliseconds Vbucket 199 2016-04-07T03:50:42.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87870 Milliseconds Vbucket 199 2016-04-07T03:50:42.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87900 Milliseconds Vbucket 199 2016-04-07T03:50:42.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87930 Milliseconds Vbucket 199 2016-04-07T03:50:42.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87960 Milliseconds Vbucket 199 2016-04-07T03:50:42.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 87990 Milliseconds Vbucket 199 2016-04-07T03:50:42.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88020 Milliseconds Vbucket 199 2016-04-07T03:50:42.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88050 Milliseconds Vbucket 199 2016-04-07T03:50:42.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88080 Milliseconds Vbucket 199 2016-04-07T03:50:42.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88110 Milliseconds Vbucket 199 2016-04-07T03:50:42.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88140 Milliseconds Vbucket 199 2016-04-07T03:50:42.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88170 Milliseconds Vbucket 199 2016-04-07T03:50:42.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88200 Milliseconds Vbucket 199 2016-04-07T03:50:42.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88230 Milliseconds Vbucket 199 2016-04-07T03:50:42.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88260 Milliseconds Vbucket 199 2016-04-07T03:50:42.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88290 Milliseconds Vbucket 199 2016-04-07T03:50:42.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88320 Milliseconds Vbucket 199 2016-04-07T03:50:42.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88350 Milliseconds Vbucket 199 2016-04-07T03:50:42.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88380 Milliseconds Vbucket 199 2016-04-07T03:50:42.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88410 Milliseconds Vbucket 199 2016-04-07T03:50:42.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88440 Milliseconds Vbucket 199 2016-04-07T03:50:43.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88470 Milliseconds Vbucket 199 2016-04-07T03:50:43.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88500 Milliseconds Vbucket 199 2016-04-07T03:50:43.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88530 Milliseconds Vbucket 199 2016-04-07T03:50:43.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88560 Milliseconds Vbucket 199 2016-04-07T03:50:43.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88590 Milliseconds Vbucket 199 2016-04-07T03:50:43.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88620 Milliseconds Vbucket 199 2016-04-07T03:50:43.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88650 Milliseconds Vbucket 199 2016-04-07T03:50:43.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88680 Milliseconds Vbucket 199 2016-04-07T03:50:43.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88710 Milliseconds Vbucket 199 2016-04-07T03:50:43.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88740 Milliseconds Vbucket 199 2016-04-07T03:50:43.299-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.495µs CommitTime 547.319252ms TotalFlushTime 10m5.28016841s TotalCommitTime 41.667286707s 2016-04-07T03:50:43.299-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 15049607, 15049607, 113 committed:true 2016-04-07T03:50:43.299-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 15049607, 15049607, 113 committed:true) SnapCreateDur 558.332333ms SnapOpenDur 37.505µs 2016-04-07T03:50:43.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88770 Milliseconds Vbucket 199 2016-04-07T03:50:43.328-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15050000 QueuedCount 14765 2016-04-07T03:50:43.336-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:43.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88800 Milliseconds Vbucket 199 2016-04-07T03:50:43.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88830 Milliseconds Vbucket 199 2016-04-07T03:50:43.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88860 Milliseconds Vbucket 199 2016-04-07T03:50:43.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88890 Milliseconds Vbucket 199 2016-04-07T03:50:43.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88920 Milliseconds Vbucket 199 2016-04-07T03:50:43.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88950 Milliseconds Vbucket 199 2016-04-07T03:50:43.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 88980 Milliseconds Vbucket 199 2016-04-07T03:50:43.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89010 Milliseconds Vbucket 199 2016-04-07T03:50:43.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89040 Milliseconds Vbucket 199 2016-04-07T03:50:43.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89070 Milliseconds Vbucket 199 2016-04-07T03:50:43.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89100 Milliseconds Vbucket 199 2016-04-07T03:50:43.664-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:43.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89130 Milliseconds Vbucket 199 2016-04-07T03:50:43.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89160 Milliseconds Vbucket 199 2016-04-07T03:50:43.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89190 Milliseconds Vbucket 199 2016-04-07T03:50:43.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89220 Milliseconds Vbucket 199 2016-04-07T03:50:43.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89250 Milliseconds Vbucket 199 2016-04-07T03:50:43.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89280 Milliseconds Vbucket 199 2016-04-07T03:50:43.838-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15060000 QueuedCount 50000 2016-04-07T03:50:43.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89310 Milliseconds Vbucket 199 2016-04-07T03:50:43.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89340 Milliseconds Vbucket 199 2016-04-07T03:50:43.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89370 Milliseconds Vbucket 199 2016-04-07T03:50:43.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89400 Milliseconds Vbucket 199 2016-04-07T03:50:43.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89430 Milliseconds Vbucket 199 2016-04-07T03:50:44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89460 Milliseconds Vbucket 199 2016-04-07T03:50:44.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89490 Milliseconds Vbucket 199 2016-04-07T03:50:44.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89520 Milliseconds Vbucket 199 2016-04-07T03:50:44.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89550 Milliseconds Vbucket 199 2016-04-07T03:50:44.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89580 Milliseconds Vbucket 199 2016-04-07T03:50:44.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89610 Milliseconds Vbucket 199 2016-04-07T03:50:44.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89640 Milliseconds Vbucket 199 2016-04-07T03:50:44.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89670 Milliseconds Vbucket 199 2016-04-07T03:50:44.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89700 Milliseconds Vbucket 199 2016-04-07T03:50:44.246-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15070000 QueuedCount 50000 2016-04-07T03:50:44.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89730 Milliseconds Vbucket 199 2016-04-07T03:50:44.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89760 Milliseconds Vbucket 199 2016-04-07T03:50:44.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89790 Milliseconds Vbucket 199 2016-04-07T03:50:44.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89820 Milliseconds Vbucket 199 2016-04-07T03:50:44.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89850 Milliseconds Vbucket 199 2016-04-07T03:50:44.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89880 Milliseconds Vbucket 199 2016-04-07T03:50:44.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89910 Milliseconds Vbucket 199 2016-04-07T03:50:44.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89940 Milliseconds Vbucket 199 2016-04-07T03:50:44.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 89970 Milliseconds Vbucket 199 2016-04-07T03:50:44.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90000 Milliseconds Vbucket 199 2016-04-07T03:50:44.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90030 Milliseconds Vbucket 199 2016-04-07T03:50:44.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90060 Milliseconds Vbucket 199 2016-04-07T03:50:44.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90090 Milliseconds Vbucket 199 2016-04-07T03:50:44.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90120 Milliseconds Vbucket 199 2016-04-07T03:50:44.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90150 Milliseconds Vbucket 199 2016-04-07T03:50:44.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90180 Milliseconds Vbucket 199 2016-04-07T03:50:44.727-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15080000 QueuedCount 50000 2016-04-07T03:50:44.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90210 Milliseconds Vbucket 199 2016-04-07T03:50:44.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90240 Milliseconds Vbucket 199 2016-04-07T03:50:44.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90270 Milliseconds Vbucket 199 2016-04-07T03:50:44.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90300 Milliseconds Vbucket 199 2016-04-07T03:50:44.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90330 Milliseconds Vbucket 199 2016-04-07T03:50:44.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90360 Milliseconds Vbucket 199 2016-04-07T03:50:44.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90390 Milliseconds Vbucket 199 2016-04-07T03:50:44.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90420 Milliseconds Vbucket 199 2016-04-07T03:50:44.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90450 Milliseconds Vbucket 199 2016-04-07T03:50:45.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90480 Milliseconds Vbucket 199 2016-04-07T03:50:45.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90510 Milliseconds Vbucket 199 2016-04-07T03:50:45.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90540 Milliseconds Vbucket 199 2016-04-07T03:50:45.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90570 Milliseconds Vbucket 199 2016-04-07T03:50:45.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90600 Milliseconds Vbucket 199 2016-04-07T03:50:45.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90630 Milliseconds Vbucket 199 2016-04-07T03:50:45.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90660 Milliseconds Vbucket 199 2016-04-07T03:50:45.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90690 Milliseconds Vbucket 199 2016-04-07T03:50:45.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90720 Milliseconds Vbucket 199 2016-04-07T03:50:45.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90750 Milliseconds Vbucket 199 2016-04-07T03:50:45.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90780 Milliseconds Vbucket 199 2016-04-07T03:50:45.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90810 Milliseconds Vbucket 199 2016-04-07T03:50:45.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90840 Milliseconds Vbucket 199 2016-04-07T03:50:45.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90870 Milliseconds Vbucket 199 2016-04-07T03:50:45.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90900 Milliseconds Vbucket 199 2016-04-07T03:50:45.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90930 Milliseconds Vbucket 199 2016-04-07T03:50:45.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90960 Milliseconds Vbucket 199 2016-04-07T03:50:45.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 90990 Milliseconds Vbucket 199 2016-04-07T03:50:45.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91020 Milliseconds Vbucket 199 2016-04-07T03:50:45.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91050 Milliseconds Vbucket 199 2016-04-07T03:50:45.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91080 Milliseconds Vbucket 199 2016-04-07T03:50:45.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91110 Milliseconds Vbucket 199 2016-04-07T03:50:45.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91140 Milliseconds Vbucket 199 2016-04-07T03:50:45.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91170 Milliseconds Vbucket 199 2016-04-07T03:50:45.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91200 Milliseconds Vbucket 199 2016-04-07T03:50:45.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91230 Milliseconds Vbucket 199 2016-04-07T03:50:45.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91260 Milliseconds Vbucket 199 2016-04-07T03:50:45.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91290 Milliseconds Vbucket 199 2016-04-07T03:50:45.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91320 Milliseconds Vbucket 199 2016-04-07T03:50:45.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91350 Milliseconds Vbucket 199 2016-04-07T03:50:45.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91380 Milliseconds Vbucket 199 2016-04-07T03:50:45.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91410 Milliseconds Vbucket 199 2016-04-07T03:50:45.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91440 Milliseconds Vbucket 199 2016-04-07T03:50:46.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91470 Milliseconds Vbucket 199 2016-04-07T03:50:46.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91500 Milliseconds Vbucket 199 2016-04-07T03:50:46.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91530 Milliseconds Vbucket 199 2016-04-07T03:50:46.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91560 Milliseconds Vbucket 199 2016-04-07T03:50:46.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91590 Milliseconds Vbucket 199 2016-04-07T03:50:46.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91620 Milliseconds Vbucket 199 2016-04-07T03:50:46.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91650 Milliseconds Vbucket 199 2016-04-07T03:50:46.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91680 Milliseconds Vbucket 199 2016-04-07T03:50:46.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91710 Milliseconds Vbucket 199 2016-04-07T03:50:46.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91740 Milliseconds Vbucket 199 2016-04-07T03:50:46.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91770 Milliseconds Vbucket 199 2016-04-07T03:50:46.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91800 Milliseconds Vbucket 199 2016-04-07T03:50:46.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91830 Milliseconds Vbucket 199 2016-04-07T03:50:46.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91860 Milliseconds Vbucket 199 2016-04-07T03:50:46.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91890 Milliseconds Vbucket 199 2016-04-07T03:50:46.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91920 Milliseconds Vbucket 199 2016-04-07T03:50:46.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91950 Milliseconds Vbucket 199 2016-04-07T03:50:46.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 91980 Milliseconds Vbucket 199 2016-04-07T03:50:46.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92010 Milliseconds Vbucket 199 2016-04-07T03:50:46.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92040 Milliseconds Vbucket 199 2016-04-07T03:50:46.602-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:46.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92070 Milliseconds Vbucket 199 2016-04-07T03:50:46.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92100 Milliseconds Vbucket 199 2016-04-07T03:50:46.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92130 Milliseconds Vbucket 199 2016-04-07T03:50:46.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92160 Milliseconds Vbucket 199 2016-04-07T03:50:46.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92190 Milliseconds Vbucket 199 2016-04-07T03:50:46.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92220 Milliseconds Vbucket 199 2016-04-07T03:50:46.769-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15090000 QueuedCount 50000 2016-04-07T03:50:46.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92250 Milliseconds Vbucket 199 2016-04-07T03:50:46.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92280 Milliseconds Vbucket 199 2016-04-07T03:50:46.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92310 Milliseconds Vbucket 199 2016-04-07T03:50:46.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92340 Milliseconds Vbucket 199 2016-04-07T03:50:46.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92370 Milliseconds Vbucket 199 2016-04-07T03:50:46.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92400 Milliseconds Vbucket 199 2016-04-07T03:50:46.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92430 Milliseconds Vbucket 199 2016-04-07T03:50:47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92460 Milliseconds Vbucket 199 2016-04-07T03:50:47.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92490 Milliseconds Vbucket 199 2016-04-07T03:50:47.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92520 Milliseconds Vbucket 199 2016-04-07T03:50:47.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92550 Milliseconds Vbucket 199 2016-04-07T03:50:47.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92580 Milliseconds Vbucket 199 2016-04-07T03:50:47.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92610 Milliseconds Vbucket 199 2016-04-07T03:50:47.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92640 Milliseconds Vbucket 199 2016-04-07T03:50:47.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92670 Milliseconds Vbucket 199 2016-04-07T03:50:47.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92700 Milliseconds Vbucket 199 2016-04-07T03:50:47.252-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15100000 QueuedCount 50000 2016-04-07T03:50:47.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92730 Milliseconds Vbucket 199 2016-04-07T03:50:47.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92760 Milliseconds Vbucket 199 2016-04-07T03:50:47.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92790 Milliseconds Vbucket 199 2016-04-07T03:50:47.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92820 Milliseconds Vbucket 199 2016-04-07T03:50:47.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92850 Milliseconds Vbucket 199 2016-04-07T03:50:47.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92880 Milliseconds Vbucket 199 2016-04-07T03:50:47.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92910 Milliseconds Vbucket 199 2016-04-07T03:50:47.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92940 Milliseconds Vbucket 199 2016-04-07T03:50:47.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 92970 Milliseconds Vbucket 199 2016-04-07T03:50:47.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93000 Milliseconds Vbucket 199 2016-04-07T03:50:47.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93030 Milliseconds Vbucket 199 2016-04-07T03:50:47.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93060 Milliseconds Vbucket 199 2016-04-07T03:50:47.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93090 Milliseconds Vbucket 199 2016-04-07T03:50:47.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93120 Milliseconds Vbucket 199 2016-04-07T03:50:47.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93150 Milliseconds Vbucket 199 2016-04-07T03:50:47.704-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15110000 QueuedCount 50000 2016-04-07T03:50:47.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93180 Milliseconds Vbucket 199 2016-04-07T03:50:47.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93210 Milliseconds Vbucket 199 2016-04-07T03:50:47.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93240 Milliseconds Vbucket 199 2016-04-07T03:50:47.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93270 Milliseconds Vbucket 199 2016-04-07T03:50:47.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93300 Milliseconds Vbucket 199 2016-04-07T03:50:47.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93330 Milliseconds Vbucket 199 2016-04-07T03:50:47.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93360 Milliseconds Vbucket 199 2016-04-07T03:50:47.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93390 Milliseconds Vbucket 199 2016-04-07T03:50:47.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93420 Milliseconds Vbucket 199 2016-04-07T03:50:47.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93450 Milliseconds Vbucket 199 2016-04-07T03:50:48.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93480 Milliseconds Vbucket 199 2016-04-07T03:50:48.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93510 Milliseconds Vbucket 199 2016-04-07T03:50:48.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93540 Milliseconds Vbucket 199 2016-04-07T03:50:48.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93570 Milliseconds Vbucket 199 2016-04-07T03:50:48.11-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15120000 QueuedCount 46129 2016-04-07T03:50:48.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93600 Milliseconds Vbucket 199 2016-04-07T03:50:48.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93630 Milliseconds Vbucket 199 2016-04-07T03:50:48.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93660 Milliseconds Vbucket 199 2016-04-07T03:50:48.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93690 Milliseconds Vbucket 199 2016-04-07T03:50:48.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93720 Milliseconds Vbucket 199 2016-04-07T03:50:48.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93750 Milliseconds Vbucket 199 2016-04-07T03:50:48.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93780 Milliseconds Vbucket 199 2016-04-07T03:50:48.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93810 Milliseconds Vbucket 199 2016-04-07T03:50:48.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93840 Milliseconds Vbucket 199 2016-04-07T03:50:48.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93870 Milliseconds Vbucket 199 2016-04-07T03:50:48.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93900 Milliseconds Vbucket 199 2016-04-07T03:50:48.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93930 Milliseconds Vbucket 199 2016-04-07T03:50:48.484-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15130000 QueuedCount 36129 2016-04-07T03:50:48.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93960 Milliseconds Vbucket 199 2016-04-07T03:50:48.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 93990 Milliseconds Vbucket 199 2016-04-07T03:50:48.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94020 Milliseconds Vbucket 199 2016-04-07T03:50:48.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94050 Milliseconds Vbucket 199 2016-04-07T03:50:48.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94080 Milliseconds Vbucket 199 2016-04-07T03:50:48.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94110 Milliseconds Vbucket 199 2016-04-07T03:50:48.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94140 Milliseconds Vbucket 199 2016-04-07T03:50:48.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94170 Milliseconds Vbucket 199 2016-04-07T03:50:48.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94200 Milliseconds Vbucket 199 2016-04-07T03:50:48.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94230 Milliseconds Vbucket 199 2016-04-07T03:50:48.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94260 Milliseconds Vbucket 199 2016-04-07T03:50:48.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94290 Milliseconds Vbucket 199 2016-04-07T03:50:48.838-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15140000 QueuedCount 26129 2016-04-07T03:50:48.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94320 Milliseconds Vbucket 199 2016-04-07T03:50:48.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94350 Milliseconds Vbucket 199 2016-04-07T03:50:48.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94380 Milliseconds Vbucket 199 2016-04-07T03:50:48.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94410 Milliseconds Vbucket 199 2016-04-07T03:50:48.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94440 Milliseconds Vbucket 199 2016-04-07T03:50:49.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94470 Milliseconds Vbucket 199 2016-04-07T03:50:49.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94500 Milliseconds Vbucket 199 2016-04-07T03:50:49.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94530 Milliseconds Vbucket 199 2016-04-07T03:50:49.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94560 Milliseconds Vbucket 199 2016-04-07T03:50:49.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94590 Milliseconds Vbucket 199 2016-04-07T03:50:49.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94620 Milliseconds Vbucket 199 2016-04-07T03:50:49.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94650 Milliseconds Vbucket 199 2016-04-07T03:50:49.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94680 Milliseconds Vbucket 199 2016-04-07T03:50:49.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94710 Milliseconds Vbucket 199 2016-04-07T03:50:49.27-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15150000 QueuedCount 16129 2016-04-07T03:50:49.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94740 Milliseconds Vbucket 199 2016-04-07T03:50:49.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94770 Milliseconds Vbucket 199 2016-04-07T03:50:49.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94800 Milliseconds Vbucket 199 2016-04-07T03:50:49.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94830 Milliseconds Vbucket 199 2016-04-07T03:50:49.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94860 Milliseconds Vbucket 199 2016-04-07T03:50:49.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94890 Milliseconds Vbucket 199 2016-04-07T03:50:49.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94920 Milliseconds Vbucket 199 2016-04-07T03:50:49.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94950 Milliseconds Vbucket 199 2016-04-07T03:50:49.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 94980 Milliseconds Vbucket 199 2016-04-07T03:50:49.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95010 Milliseconds Vbucket 199 2016-04-07T03:50:49.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95040 Milliseconds Vbucket 199 2016-04-07T03:50:49.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95070 Milliseconds Vbucket 199 2016-04-07T03:50:49.62-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15160000 QueuedCount 6129 2016-04-07T03:50:49.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95100 Milliseconds Vbucket 199 2016-04-07T03:50:49.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95130 Milliseconds Vbucket 199 2016-04-07T03:50:49.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95160 Milliseconds Vbucket 199 2016-04-07T03:50:49.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95190 Milliseconds Vbucket 199 2016-04-07T03:50:49.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95220 Milliseconds Vbucket 199 2016-04-07T03:50:49.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95250 Milliseconds Vbucket 199 2016-04-07T03:50:49.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95280 Milliseconds Vbucket 199 2016-04-07T03:50:49.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95310 Milliseconds Vbucket 199 2016-04-07T03:50:49.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95340 Milliseconds Vbucket 199 2016-04-07T03:50:49.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95370 Milliseconds Vbucket 199 2016-04-07T03:50:49.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95400 Milliseconds Vbucket 199 2016-04-07T03:50:49.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95430 Milliseconds Vbucket 199 2016-04-07T03:50:50-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95460 Milliseconds Vbucket 199 2016-04-07T03:50:50.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95490 Milliseconds Vbucket 199 2016-04-07T03:50:50.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95520 Milliseconds Vbucket 199 2016-04-07T03:50:50.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95550 Milliseconds Vbucket 199 2016-04-07T03:50:50.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95580 Milliseconds Vbucket 199 2016-04-07T03:50:50.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95610 Milliseconds Vbucket 199 2016-04-07T03:50:50.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95640 Milliseconds Vbucket 199 2016-04-07T03:50:50.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95670 Milliseconds Vbucket 199 2016-04-07T03:50:50.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95700 Milliseconds Vbucket 199 2016-04-07T03:50:50.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95730 Milliseconds Vbucket 199 2016-04-07T03:50:50.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95760 Milliseconds Vbucket 199 2016-04-07T03:50:50.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95790 Milliseconds Vbucket 199 2016-04-07T03:50:50.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95820 Milliseconds Vbucket 199 2016-04-07T03:50:50.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95850 Milliseconds Vbucket 199 2016-04-07T03:50:50.415-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.396µs CommitTime 543.127672ms TotalFlushTime 10m11.64678917s TotalCommitTime 42.210414379s 2016-04-07T03:50:50.415-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 15166129, 15166129, 114 committed:true 2016-04-07T03:50:50.415-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 15166129, 15166129, 114 committed:true) SnapCreateDur 555.273283ms SnapOpenDur 54.483µs 2016-04-07T03:50:50.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95880 Milliseconds Vbucket 199 2016-04-07T03:50:50.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95910 Milliseconds Vbucket 199 2016-04-07T03:50:50.456-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:50.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95940 Milliseconds Vbucket 199 2016-04-07T03:50:50.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 95970 Milliseconds Vbucket 199 2016-04-07T03:50:50.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96000 Milliseconds Vbucket 199 2016-04-07T03:50:50.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96030 Milliseconds Vbucket 199 2016-04-07T03:50:50.576-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15170000 QueuedCount 50000 2016-04-07T03:50:50.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96060 Milliseconds Vbucket 199 2016-04-07T03:50:50.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96090 Milliseconds Vbucket 199 2016-04-07T03:50:50.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96120 Milliseconds Vbucket 199 2016-04-07T03:50:50.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96150 Milliseconds Vbucket 199 2016-04-07T03:50:50.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96180 Milliseconds Vbucket 199 2016-04-07T03:50:50.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96210 Milliseconds Vbucket 199 2016-04-07T03:50:50.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96240 Milliseconds Vbucket 199 2016-04-07T03:50:50.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96270 Milliseconds Vbucket 199 2016-04-07T03:50:50.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96300 Milliseconds Vbucket 199 2016-04-07T03:50:50.864-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:50.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96330 Milliseconds Vbucket 199 2016-04-07T03:50:50.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96360 Milliseconds Vbucket 199 2016-04-07T03:50:50.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96390 Milliseconds Vbucket 199 2016-04-07T03:50:50.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96420 Milliseconds Vbucket 199 2016-04-07T03:50:50.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96450 Milliseconds Vbucket 199 2016-04-07T03:50:51.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96480 Milliseconds Vbucket 199 2016-04-07T03:50:51.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96510 Milliseconds Vbucket 199 2016-04-07T03:50:51.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96540 Milliseconds Vbucket 199 2016-04-07T03:50:51.092-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15180000 QueuedCount 50000 2016-04-07T03:50:51.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96570 Milliseconds Vbucket 199 2016-04-07T03:50:51.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96600 Milliseconds Vbucket 199 2016-04-07T03:50:51.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96630 Milliseconds Vbucket 199 2016-04-07T03:50:51.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96660 Milliseconds Vbucket 199 2016-04-07T03:50:51.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96690 Milliseconds Vbucket 199 2016-04-07T03:50:51.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96720 Milliseconds Vbucket 199 2016-04-07T03:50:51.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96750 Milliseconds Vbucket 199 2016-04-07T03:50:51.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96780 Milliseconds Vbucket 199 2016-04-07T03:50:51.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96810 Milliseconds Vbucket 199 2016-04-07T03:50:51.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96840 Milliseconds Vbucket 199 2016-04-07T03:50:51.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96870 Milliseconds Vbucket 199 2016-04-07T03:50:51.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96900 Milliseconds Vbucket 199 2016-04-07T03:50:51.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96930 Milliseconds Vbucket 199 2016-04-07T03:50:51.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96960 Milliseconds Vbucket 199 2016-04-07T03:50:51.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 96990 Milliseconds Vbucket 199 2016-04-07T03:50:51.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97020 Milliseconds Vbucket 199 2016-04-07T03:50:51.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97050 Milliseconds Vbucket 199 2016-04-07T03:50:51.601-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15190000 QueuedCount 50000 2016-04-07T03:50:51.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97080 Milliseconds Vbucket 199 2016-04-07T03:50:51.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97110 Milliseconds Vbucket 199 2016-04-07T03:50:51.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97140 Milliseconds Vbucket 199 2016-04-07T03:50:51.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97170 Milliseconds Vbucket 199 2016-04-07T03:50:51.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97200 Milliseconds Vbucket 199 2016-04-07T03:50:51.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97230 Milliseconds Vbucket 199 2016-04-07T03:50:51.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97260 Milliseconds Vbucket 199 2016-04-07T03:50:51.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97290 Milliseconds Vbucket 199 2016-04-07T03:50:51.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97320 Milliseconds Vbucket 199 2016-04-07T03:50:51.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97350 Milliseconds Vbucket 199 2016-04-07T03:50:51.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97380 Milliseconds Vbucket 199 2016-04-07T03:50:51.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97410 Milliseconds Vbucket 199 2016-04-07T03:50:51.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97440 Milliseconds Vbucket 199 2016-04-07T03:50:52.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97470 Milliseconds Vbucket 199 2016-04-07T03:50:52.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97500 Milliseconds Vbucket 199 2016-04-07T03:50:52.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97530 Milliseconds Vbucket 199 2016-04-07T03:50:52.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97560 Milliseconds Vbucket 199 2016-04-07T03:50:52.115-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15200000 QueuedCount 50000 2016-04-07T03:50:52.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97590 Milliseconds Vbucket 199 2016-04-07T03:50:52.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97620 Milliseconds Vbucket 199 2016-04-07T03:50:52.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97650 Milliseconds Vbucket 199 2016-04-07T03:50:52.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97680 Milliseconds Vbucket 199 2016-04-07T03:50:52.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97710 Milliseconds Vbucket 199 2016-04-07T03:50:52.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97740 Milliseconds Vbucket 199 2016-04-07T03:50:52.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97770 Milliseconds Vbucket 199 2016-04-07T03:50:52.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97800 Milliseconds Vbucket 199 2016-04-07T03:50:52.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97830 Milliseconds Vbucket 199 2016-04-07T03:50:52.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97860 Milliseconds Vbucket 199 2016-04-07T03:50:52.419-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:52.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97890 Milliseconds Vbucket 199 2016-04-07T03:50:52.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97920 Milliseconds Vbucket 199 2016-04-07T03:50:52.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97950 Milliseconds Vbucket 199 2016-04-07T03:50:52.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 97980 Milliseconds Vbucket 199 2016-04-07T03:50:52.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98010 Milliseconds Vbucket 199 2016-04-07T03:50:52.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98040 Milliseconds Vbucket 199 2016-04-07T03:50:52.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98070 Milliseconds Vbucket 199 2016-04-07T03:50:52.625-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15210000 QueuedCount 50000 2016-04-07T03:50:52.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98100 Milliseconds Vbucket 199 2016-04-07T03:50:52.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98130 Milliseconds Vbucket 199 2016-04-07T03:50:52.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98160 Milliseconds Vbucket 199 2016-04-07T03:50:52.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98190 Milliseconds Vbucket 199 2016-04-07T03:50:52.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98220 Milliseconds Vbucket 199 2016-04-07T03:50:52.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98250 Milliseconds Vbucket 199 2016-04-07T03:50:52.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98280 Milliseconds Vbucket 199 2016-04-07T03:50:52.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98310 Milliseconds Vbucket 199 2016-04-07T03:50:52.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98340 Milliseconds Vbucket 199 2016-04-07T03:50:52.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98370 Milliseconds Vbucket 199 2016-04-07T03:50:52.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98400 Milliseconds Vbucket 199 2016-04-07T03:50:52.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98430 Milliseconds Vbucket 199 2016-04-07T03:50:53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98460 Milliseconds Vbucket 199 2016-04-07T03:50:53.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98490 Milliseconds Vbucket 199 2016-04-07T03:50:53.042-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15220000 QueuedCount 50000 2016-04-07T03:50:53.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98520 Milliseconds Vbucket 199 2016-04-07T03:50:53.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98550 Milliseconds Vbucket 199 2016-04-07T03:50:53.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98580 Milliseconds Vbucket 199 2016-04-07T03:50:53.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98610 Milliseconds Vbucket 199 2016-04-07T03:50:53.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98640 Milliseconds Vbucket 199 2016-04-07T03:50:53.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98670 Milliseconds Vbucket 199 2016-04-07T03:50:53.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98700 Milliseconds Vbucket 199 2016-04-07T03:50:53.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98730 Milliseconds Vbucket 199 2016-04-07T03:50:53.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98760 Milliseconds Vbucket 199 2016-04-07T03:50:53.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98790 Milliseconds Vbucket 199 2016-04-07T03:50:53.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98820 Milliseconds Vbucket 199 2016-04-07T03:50:53.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98850 Milliseconds Vbucket 199 2016-04-07T03:50:53.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98880 Milliseconds Vbucket 199 2016-04-07T03:50:53.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98910 Milliseconds Vbucket 199 2016-04-07T03:50:53.462-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15230000 QueuedCount 50000 2016-04-07T03:50:53.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98940 Milliseconds Vbucket 199 2016-04-07T03:50:53.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 98970 Milliseconds Vbucket 199 2016-04-07T03:50:53.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99000 Milliseconds Vbucket 199 2016-04-07T03:50:53.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99030 Milliseconds Vbucket 199 2016-04-07T03:50:53.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99060 Milliseconds Vbucket 199 2016-04-07T03:50:53.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99090 Milliseconds Vbucket 199 2016-04-07T03:50:53.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99120 Milliseconds Vbucket 199 2016-04-07T03:50:53.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99150 Milliseconds Vbucket 199 2016-04-07T03:50:53.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99180 Milliseconds Vbucket 199 2016-04-07T03:50:53.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99210 Milliseconds Vbucket 199 2016-04-07T03:50:53.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99240 Milliseconds Vbucket 199 2016-04-07T03:50:53.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99270 Milliseconds Vbucket 199 2016-04-07T03:50:53.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99300 Milliseconds Vbucket 199 2016-04-07T03:50:53.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99330 Milliseconds Vbucket 199 2016-04-07T03:50:53.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99360 Milliseconds Vbucket 199 2016-04-07T03:50:53.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99390 Milliseconds Vbucket 199 2016-04-07T03:50:53.948-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15240000 QueuedCount 47579 2016-04-07T03:50:53.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99420 Milliseconds Vbucket 199 2016-04-07T03:50:53.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99450 Milliseconds Vbucket 199 2016-04-07T03:50:54.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99480 Milliseconds Vbucket 199 2016-04-07T03:50:54.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99510 Milliseconds Vbucket 199 2016-04-07T03:50:54.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99540 Milliseconds Vbucket 199 2016-04-07T03:50:54.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99570 Milliseconds Vbucket 199 2016-04-07T03:50:54.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99600 Milliseconds Vbucket 199 2016-04-07T03:50:54.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99630 Milliseconds Vbucket 199 2016-04-07T03:50:54.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99660 Milliseconds Vbucket 199 2016-04-07T03:50:54.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99690 Milliseconds Vbucket 199 2016-04-07T03:50:54.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99720 Milliseconds Vbucket 199 2016-04-07T03:50:54.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99750 Milliseconds Vbucket 199 2016-04-07T03:50:54.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99780 Milliseconds Vbucket 199 2016-04-07T03:50:54.338-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15250000 QueuedCount 37579 2016-04-07T03:50:54.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99810 Milliseconds Vbucket 199 2016-04-07T03:50:54.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99840 Milliseconds Vbucket 199 2016-04-07T03:50:54.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99870 Milliseconds Vbucket 199 2016-04-07T03:50:54.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99900 Milliseconds Vbucket 199 2016-04-07T03:50:54.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99930 Milliseconds Vbucket 199 2016-04-07T03:50:54.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99960 Milliseconds Vbucket 199 2016-04-07T03:50:54.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 99990 Milliseconds Vbucket 199 2016-04-07T03:50:54.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100020 Milliseconds Vbucket 199 2016-04-07T03:50:54.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100050 Milliseconds Vbucket 199 2016-04-07T03:50:54.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100080 Milliseconds Vbucket 199 2016-04-07T03:50:54.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100110 Milliseconds Vbucket 199 2016-04-07T03:50:54.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100140 Milliseconds Vbucket 199 2016-04-07T03:50:54.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100170 Milliseconds Vbucket 199 2016-04-07T03:50:54.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100200 Milliseconds Vbucket 199 2016-04-07T03:50:54.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100230 Milliseconds Vbucket 199 2016-04-07T03:50:54.788-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15260000 QueuedCount 27579 2016-04-07T03:50:54.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100260 Milliseconds Vbucket 199 2016-04-07T03:50:54.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100290 Milliseconds Vbucket 199 2016-04-07T03:50:54.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100320 Milliseconds Vbucket 199 2016-04-07T03:50:54.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100350 Milliseconds Vbucket 199 2016-04-07T03:50:54.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100380 Milliseconds Vbucket 199 2016-04-07T03:50:54.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100410 Milliseconds Vbucket 199 2016-04-07T03:50:54.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100440 Milliseconds Vbucket 199 2016-04-07T03:50:55.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100470 Milliseconds Vbucket 199 2016-04-07T03:50:55.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100500 Milliseconds Vbucket 199 2016-04-07T03:50:55.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100530 Milliseconds Vbucket 199 2016-04-07T03:50:55.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100560 Milliseconds Vbucket 199 2016-04-07T03:50:55.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100590 Milliseconds Vbucket 199 2016-04-07T03:50:55.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100620 Milliseconds Vbucket 199 2016-04-07T03:50:55.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100650 Milliseconds Vbucket 199 2016-04-07T03:50:55.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100680 Milliseconds Vbucket 199 2016-04-07T03:50:55.235-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15270000 QueuedCount 17579 2016-04-07T03:50:55.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100710 Milliseconds Vbucket 199 2016-04-07T03:50:55.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100740 Milliseconds Vbucket 199 2016-04-07T03:50:55.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100770 Milliseconds Vbucket 199 2016-04-07T03:50:55.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100800 Milliseconds Vbucket 199 2016-04-07T03:50:55.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100830 Milliseconds Vbucket 199 2016-04-07T03:50:55.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100860 Milliseconds Vbucket 199 2016-04-07T03:50:55.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100890 Milliseconds Vbucket 199 2016-04-07T03:50:55.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100920 Milliseconds Vbucket 199 2016-04-07T03:50:55.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100950 Milliseconds Vbucket 199 2016-04-07T03:50:55.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 100980 Milliseconds Vbucket 199 2016-04-07T03:50:55.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101010 Milliseconds Vbucket 199 2016-04-07T03:50:55.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101040 Milliseconds Vbucket 199 2016-04-07T03:50:55.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101070 Milliseconds Vbucket 199 2016-04-07T03:50:55.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101100 Milliseconds Vbucket 199 2016-04-07T03:50:55.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101130 Milliseconds Vbucket 199 2016-04-07T03:50:55.685-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15280000 QueuedCount 7579 2016-04-07T03:50:55.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101160 Milliseconds Vbucket 199 2016-04-07T03:50:55.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101190 Milliseconds Vbucket 199 2016-04-07T03:50:55.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101220 Milliseconds Vbucket 199 2016-04-07T03:50:55.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101250 Milliseconds Vbucket 199 2016-04-07T03:50:55.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101280 Milliseconds Vbucket 199 2016-04-07T03:50:55.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101310 Milliseconds Vbucket 199 2016-04-07T03:50:55.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101340 Milliseconds Vbucket 199 2016-04-07T03:50:55.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101370 Milliseconds Vbucket 199 2016-04-07T03:50:55.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101400 Milliseconds Vbucket 199 2016-04-07T03:50:55.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101430 Milliseconds Vbucket 199 2016-04-07T03:50:56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101460 Milliseconds Vbucket 199 2016-04-07T03:50:56.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101490 Milliseconds Vbucket 199 2016-04-07T03:50:56.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101520 Milliseconds Vbucket 199 2016-04-07T03:50:56.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101550 Milliseconds Vbucket 199 2016-04-07T03:50:56.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101580 Milliseconds Vbucket 199 2016-04-07T03:50:56.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101610 Milliseconds Vbucket 199 2016-04-07T03:50:56.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101640 Milliseconds Vbucket 199 2016-04-07T03:50:56.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101670 Milliseconds Vbucket 199 2016-04-07T03:50:56.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101700 Milliseconds Vbucket 199 2016-04-07T03:50:56.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101730 Milliseconds Vbucket 199 2016-04-07T03:50:56.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101760 Milliseconds Vbucket 199 2016-04-07T03:50:56.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101790 Milliseconds Vbucket 199 2016-04-07T03:50:56.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101820 Milliseconds Vbucket 199 2016-04-07T03:50:56.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101850 Milliseconds Vbucket 199 2016-04-07T03:50:56.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101880 Milliseconds Vbucket 199 2016-04-07T03:50:56.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101910 Milliseconds Vbucket 199 2016-04-07T03:50:56.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101940 Milliseconds Vbucket 199 2016-04-07T03:50:56.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 101970 Milliseconds Vbucket 199 2016-04-07T03:50:56.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102000 Milliseconds Vbucket 199 2016-04-07T03:50:56.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102030 Milliseconds Vbucket 199 2016-04-07T03:50:56.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102060 Milliseconds Vbucket 199 2016-04-07T03:50:56.606-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.939µs CommitTime 563.130102ms TotalFlushTime 10m17.05214507s TotalCommitTime 42.773544481s 2016-04-07T03:50:56.607-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 15287579, 15287579, 115 committed:true 2016-04-07T03:50:56.607-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 15287579, 15287579, 115 committed:true) SnapCreateDur 573.64658ms SnapOpenDur 40.11µs 2016-04-07T03:50:56.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102090 Milliseconds Vbucket 199 2016-04-07T03:50:56.649-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:56.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102120 Milliseconds Vbucket 199 2016-04-07T03:50:56.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102150 Milliseconds Vbucket 199 2016-04-07T03:50:56.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102180 Milliseconds Vbucket 199 2016-04-07T03:50:56.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102210 Milliseconds Vbucket 199 2016-04-07T03:50:56.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102240 Milliseconds Vbucket 199 2016-04-07T03:50:56.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102270 Milliseconds Vbucket 199 2016-04-07T03:50:56.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102300 Milliseconds Vbucket 199 2016-04-07T03:50:56.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102330 Milliseconds Vbucket 199 2016-04-07T03:50:56.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102360 Milliseconds Vbucket 199 2016-04-07T03:50:56.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102390 Milliseconds Vbucket 199 2016-04-07T03:50:56.946-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15290000 QueuedCount 50000 2016-04-07T03:50:56.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102420 Milliseconds Vbucket 199 2016-04-07T03:50:56.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102450 Milliseconds Vbucket 199 2016-04-07T03:50:57.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102480 Milliseconds Vbucket 199 2016-04-07T03:50:57.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102510 Milliseconds Vbucket 199 2016-04-07T03:50:57.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102540 Milliseconds Vbucket 199 2016-04-07T03:50:57.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102570 Milliseconds Vbucket 199 2016-04-07T03:50:57.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102600 Milliseconds Vbucket 199 2016-04-07T03:50:57.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102630 Milliseconds Vbucket 199 2016-04-07T03:50:57.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102660 Milliseconds Vbucket 199 2016-04-07T03:50:57.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102690 Milliseconds Vbucket 199 2016-04-07T03:50:57.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102720 Milliseconds Vbucket 199 2016-04-07T03:50:57.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102750 Milliseconds Vbucket 199 2016-04-07T03:50:57.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102780 Milliseconds Vbucket 199 2016-04-07T03:50:57.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102810 Milliseconds Vbucket 199 2016-04-07T03:50:57.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102840 Milliseconds Vbucket 199 2016-04-07T03:50:57.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102870 Milliseconds Vbucket 199 2016-04-07T03:50:57.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102900 Milliseconds Vbucket 199 2016-04-07T03:50:57.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102930 Milliseconds Vbucket 199 2016-04-07T03:50:57.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102960 Milliseconds Vbucket 199 2016-04-07T03:50:57.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 102990 Milliseconds Vbucket 199 2016-04-07T03:50:57.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103020 Milliseconds Vbucket 199 2016-04-07T03:50:57.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103050 Milliseconds Vbucket 199 2016-04-07T03:50:57.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103080 Milliseconds Vbucket 199 2016-04-07T03:50:57.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103110 Milliseconds Vbucket 199 2016-04-07T03:50:57.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103140 Milliseconds Vbucket 199 2016-04-07T03:50:57.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103170 Milliseconds Vbucket 199 2016-04-07T03:50:57.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103200 Milliseconds Vbucket 199 2016-04-07T03:50:57.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103230 Milliseconds Vbucket 199 2016-04-07T03:50:57.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103260 Milliseconds Vbucket 199 2016-04-07T03:50:57.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103290 Milliseconds Vbucket 199 2016-04-07T03:50:57.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103320 Milliseconds Vbucket 199 2016-04-07T03:50:57.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103350 Milliseconds Vbucket 199 2016-04-07T03:50:57.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103380 Milliseconds Vbucket 199 2016-04-07T03:50:57.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103410 Milliseconds Vbucket 199 2016-04-07T03:50:57.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103440 Milliseconds Vbucket 199 2016-04-07T03:50:58.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103470 Milliseconds Vbucket 199 2016-04-07T03:50:58.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103500 Milliseconds Vbucket 199 2016-04-07T03:50:58.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103530 Milliseconds Vbucket 199 2016-04-07T03:50:58.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103560 Milliseconds Vbucket 199 2016-04-07T03:50:58.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103590 Milliseconds Vbucket 199 2016-04-07T03:50:58.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103620 Milliseconds Vbucket 199 2016-04-07T03:50:58.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103650 Milliseconds Vbucket 199 2016-04-07T03:50:58.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103680 Milliseconds Vbucket 199 2016-04-07T03:50:58.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103710 Milliseconds Vbucket 199 2016-04-07T03:50:58.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103740 Milliseconds Vbucket 199 2016-04-07T03:50:58.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103770 Milliseconds Vbucket 199 2016-04-07T03:50:58.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103800 Milliseconds Vbucket 199 2016-04-07T03:50:58.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103830 Milliseconds Vbucket 199 2016-04-07T03:50:58.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103860 Milliseconds Vbucket 199 2016-04-07T03:50:58.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103890 Milliseconds Vbucket 199 2016-04-07T03:50:58.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103920 Milliseconds Vbucket 199 2016-04-07T03:50:58.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103950 Milliseconds Vbucket 199 2016-04-07T03:50:58.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 103980 Milliseconds Vbucket 199 2016-04-07T03:50:58.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104010 Milliseconds Vbucket 199 2016-04-07T03:50:58.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104040 Milliseconds Vbucket 199 2016-04-07T03:50:58.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104070 Milliseconds Vbucket 199 2016-04-07T03:50:58.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104100 Milliseconds Vbucket 199 2016-04-07T03:50:58.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104130 Milliseconds Vbucket 199 2016-04-07T03:50:58.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104160 Milliseconds Vbucket 199 2016-04-07T03:50:58.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104190 Milliseconds Vbucket 199 2016-04-07T03:50:58.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104220 Milliseconds Vbucket 199 2016-04-07T03:50:58.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104250 Milliseconds Vbucket 199 2016-04-07T03:50:58.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104280 Milliseconds Vbucket 199 2016-04-07T03:50:58.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104310 Milliseconds Vbucket 199 2016-04-07T03:50:58.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104340 Milliseconds Vbucket 199 2016-04-07T03:50:58.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104370 Milliseconds Vbucket 199 2016-04-07T03:50:58.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104400 Milliseconds Vbucket 199 2016-04-07T03:50:58.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104430 Milliseconds Vbucket 199 2016-04-07T03:50:59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104460 Milliseconds Vbucket 199 2016-04-07T03:50:59.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104490 Milliseconds Vbucket 199 2016-04-07T03:50:59.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104520 Milliseconds Vbucket 199 2016-04-07T03:50:59.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104550 Milliseconds Vbucket 199 2016-04-07T03:50:59.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104580 Milliseconds Vbucket 199 2016-04-07T03:50:59.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104610 Milliseconds Vbucket 199 2016-04-07T03:50:59.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104640 Milliseconds Vbucket 199 2016-04-07T03:50:59.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104670 Milliseconds Vbucket 199 2016-04-07T03:50:59.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104700 Milliseconds Vbucket 199 2016-04-07T03:50:59.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104730 Milliseconds Vbucket 199 2016-04-07T03:50:59.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104760 Milliseconds Vbucket 199 2016-04-07T03:50:59.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104790 Milliseconds Vbucket 199 2016-04-07T03:50:59.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104820 Milliseconds Vbucket 199 2016-04-07T03:50:59.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104850 Milliseconds Vbucket 199 2016-04-07T03:50:59.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104880 Milliseconds Vbucket 199 2016-04-07T03:50:59.424-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15300000 QueuedCount 50000 2016-04-07T03:50:59.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104910 Milliseconds Vbucket 199 2016-04-07T03:50:59.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104940 Milliseconds Vbucket 199 2016-04-07T03:50:59.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 104970 Milliseconds Vbucket 199 2016-04-07T03:50:59.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105000 Milliseconds Vbucket 199 2016-04-07T03:50:59.565-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:50:59.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105030 Milliseconds Vbucket 199 2016-04-07T03:50:59.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105060 Milliseconds Vbucket 199 2016-04-07T03:50:59.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105090 Milliseconds Vbucket 199 2016-04-07T03:50:59.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105120 Milliseconds Vbucket 199 2016-04-07T03:50:59.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105150 Milliseconds Vbucket 199 2016-04-07T03:50:59.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105180 Milliseconds Vbucket 199 2016-04-07T03:50:59.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105210 Milliseconds Vbucket 199 2016-04-07T03:50:59.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105240 Milliseconds Vbucket 199 2016-04-07T03:50:59.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105270 Milliseconds Vbucket 199 2016-04-07T03:50:59.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105300 Milliseconds Vbucket 199 2016-04-07T03:50:59.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105330 Milliseconds Vbucket 199 2016-04-07T03:50:59.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105360 Milliseconds Vbucket 199 2016-04-07T03:50:59.911-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15310000 QueuedCount 50000 2016-04-07T03:50:59.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105390 Milliseconds Vbucket 199 2016-04-07T03:50:59.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105420 Milliseconds Vbucket 199 2016-04-07T03:50:59.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105450 Milliseconds Vbucket 199 2016-04-07T03:51:00.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105480 Milliseconds Vbucket 199 2016-04-07T03:51:00.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105510 Milliseconds Vbucket 199 2016-04-07T03:51:00.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105540 Milliseconds Vbucket 199 2016-04-07T03:51:00.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105570 Milliseconds Vbucket 199 2016-04-07T03:51:00.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105600 Milliseconds Vbucket 199 2016-04-07T03:51:00.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105630 Milliseconds Vbucket 199 2016-04-07T03:51:00.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105660 Milliseconds Vbucket 199 2016-04-07T03:51:00.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105690 Milliseconds Vbucket 199 2016-04-07T03:51:00.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105720 Milliseconds Vbucket 199 2016-04-07T03:51:00.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105750 Milliseconds Vbucket 199 2016-04-07T03:51:00.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105780 Milliseconds Vbucket 199 2016-04-07T03:51:00.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105810 Milliseconds Vbucket 199 2016-04-07T03:51:00.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105840 Milliseconds Vbucket 199 2016-04-07T03:51:00.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105870 Milliseconds Vbucket 199 2016-04-07T03:51:00.426-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15320000 QueuedCount 50000 2016-04-07T03:51:00.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105900 Milliseconds Vbucket 199 2016-04-07T03:51:00.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105930 Milliseconds Vbucket 199 2016-04-07T03:51:00.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105960 Milliseconds Vbucket 199 2016-04-07T03:51:00.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 105990 Milliseconds Vbucket 199 2016-04-07T03:51:00.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106020 Milliseconds Vbucket 199 2016-04-07T03:51:00.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106050 Milliseconds Vbucket 199 2016-04-07T03:51:00.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106080 Milliseconds Vbucket 199 2016-04-07T03:51:00.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106110 Milliseconds Vbucket 199 2016-04-07T03:51:00.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106140 Milliseconds Vbucket 199 2016-04-07T03:51:00.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106170 Milliseconds Vbucket 199 2016-04-07T03:51:00.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106200 Milliseconds Vbucket 199 2016-04-07T03:51:00.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106230 Milliseconds Vbucket 199 2016-04-07T03:51:00.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106260 Milliseconds Vbucket 199 2016-04-07T03:51:00.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106290 Milliseconds Vbucket 199 2016-04-07T03:51:00.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106320 Milliseconds Vbucket 199 2016-04-07T03:51:00.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106350 Milliseconds Vbucket 199 2016-04-07T03:51:00.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106380 Milliseconds Vbucket 199 2016-04-07T03:51:00.945-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15330000 QueuedCount 50000 2016-04-07T03:51:00.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106410 Milliseconds Vbucket 199 2016-04-07T03:51:00.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106440 Milliseconds Vbucket 199 2016-04-07T03:51:01.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106470 Milliseconds Vbucket 199 2016-04-07T03:51:01.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106500 Milliseconds Vbucket 199 2016-04-07T03:51:01.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106530 Milliseconds Vbucket 199 2016-04-07T03:51:01.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106560 Milliseconds Vbucket 199 2016-04-07T03:51:01.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106590 Milliseconds Vbucket 199 2016-04-07T03:51:01.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106620 Milliseconds Vbucket 199 2016-04-07T03:51:01.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106650 Milliseconds Vbucket 199 2016-04-07T03:51:01.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106680 Milliseconds Vbucket 199 2016-04-07T03:51:01.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106710 Milliseconds Vbucket 199 2016-04-07T03:51:01.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106740 Milliseconds Vbucket 199 2016-04-07T03:51:01.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106770 Milliseconds Vbucket 199 2016-04-07T03:51:01.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106800 Milliseconds Vbucket 199 2016-04-07T03:51:01.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106830 Milliseconds Vbucket 199 2016-04-07T03:51:01.386-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15340000 QueuedCount 43028 2016-04-07T03:51:01.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106860 Milliseconds Vbucket 199 2016-04-07T03:51:01.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106890 Milliseconds Vbucket 199 2016-04-07T03:51:01.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106920 Milliseconds Vbucket 199 2016-04-07T03:51:01.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106950 Milliseconds Vbucket 199 2016-04-07T03:51:01.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 106980 Milliseconds Vbucket 199 2016-04-07T03:51:01.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107010 Milliseconds Vbucket 199 2016-04-07T03:51:01.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107040 Milliseconds Vbucket 199 2016-04-07T03:51:01.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107070 Milliseconds Vbucket 199 2016-04-07T03:51:01.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107100 Milliseconds Vbucket 199 2016-04-07T03:51:01.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107130 Milliseconds Vbucket 199 2016-04-07T03:51:01.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107160 Milliseconds Vbucket 199 2016-04-07T03:51:01.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107190 Milliseconds Vbucket 199 2016-04-07T03:51:01.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107220 Milliseconds Vbucket 199 2016-04-07T03:51:01.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107250 Milliseconds Vbucket 199 2016-04-07T03:51:01.805-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15350000 QueuedCount 33028 2016-04-07T03:51:01.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107280 Milliseconds Vbucket 199 2016-04-07T03:51:01.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107310 Milliseconds Vbucket 199 2016-04-07T03:51:01.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107340 Milliseconds Vbucket 199 2016-04-07T03:51:01.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107370 Milliseconds Vbucket 199 2016-04-07T03:51:01.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107400 Milliseconds Vbucket 199 2016-04-07T03:51:01.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107430 Milliseconds Vbucket 199 2016-04-07T03:51:02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107460 Milliseconds Vbucket 199 2016-04-07T03:51:02.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107490 Milliseconds Vbucket 199 2016-04-07T03:51:02.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107520 Milliseconds Vbucket 199 2016-04-07T03:51:02.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107550 Milliseconds Vbucket 199 2016-04-07T03:51:02.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107580 Milliseconds Vbucket 199 2016-04-07T03:51:02.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107610 Milliseconds Vbucket 199 2016-04-07T03:51:02.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107640 Milliseconds Vbucket 199 2016-04-07T03:51:02.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107670 Milliseconds Vbucket 199 2016-04-07T03:51:02.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107700 Milliseconds Vbucket 199 2016-04-07T03:51:02.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107730 Milliseconds Vbucket 199 2016-04-07T03:51:02.276-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15360000 QueuedCount 23028 2016-04-07T03:51:02.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107760 Milliseconds Vbucket 199 2016-04-07T03:51:02.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107790 Milliseconds Vbucket 199 2016-04-07T03:51:02.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107820 Milliseconds Vbucket 199 2016-04-07T03:51:02.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107850 Milliseconds Vbucket 199 2016-04-07T03:51:02.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107880 Milliseconds Vbucket 199 2016-04-07T03:51:02.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107910 Milliseconds Vbucket 199 2016-04-07T03:51:02.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107940 Milliseconds Vbucket 199 2016-04-07T03:51:02.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 107970 Milliseconds Vbucket 199 2016-04-07T03:51:02.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108000 Milliseconds Vbucket 199 2016-04-07T03:51:02.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108030 Milliseconds Vbucket 199 2016-04-07T03:51:02.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108060 Milliseconds Vbucket 199 2016-04-07T03:51:02.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108090 Milliseconds Vbucket 199 2016-04-07T03:51:02.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108120 Milliseconds Vbucket 199 2016-04-07T03:51:02.675-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15370000 QueuedCount 13028 2016-04-07T03:51:02.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108150 Milliseconds Vbucket 199 2016-04-07T03:51:02.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108180 Milliseconds Vbucket 199 2016-04-07T03:51:02.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108210 Milliseconds Vbucket 199 2016-04-07T03:51:02.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108240 Milliseconds Vbucket 199 2016-04-07T03:51:02.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108270 Milliseconds Vbucket 199 2016-04-07T03:51:02.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108300 Milliseconds Vbucket 199 2016-04-07T03:51:02.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108330 Milliseconds Vbucket 199 2016-04-07T03:51:02.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108360 Milliseconds Vbucket 199 2016-04-07T03:51:02.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108390 Milliseconds Vbucket 199 2016-04-07T03:51:02.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108420 Milliseconds Vbucket 199 2016-04-07T03:51:02.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108450 Milliseconds Vbucket 199 2016-04-07T03:51:03.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108480 Milliseconds Vbucket 199 2016-04-07T03:51:03.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108510 Milliseconds Vbucket 199 2016-04-07T03:51:03.062-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15380000 QueuedCount 3028 2016-04-07T03:51:03.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108540 Milliseconds Vbucket 199 2016-04-07T03:51:03.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108570 Milliseconds Vbucket 199 2016-04-07T03:51:03.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108600 Milliseconds Vbucket 199 2016-04-07T03:51:03.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108630 Milliseconds Vbucket 199 2016-04-07T03:51:03.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108660 Milliseconds Vbucket 199 2016-04-07T03:51:03.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108690 Milliseconds Vbucket 199 2016-04-07T03:51:03.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108720 Milliseconds Vbucket 199 2016-04-07T03:51:03.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108750 Milliseconds Vbucket 199 2016-04-07T03:51:03.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108780 Milliseconds Vbucket 199 2016-04-07T03:51:03.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108810 Milliseconds Vbucket 199 2016-04-07T03:51:03.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108840 Milliseconds Vbucket 199 2016-04-07T03:51:03.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108870 Milliseconds Vbucket 199 2016-04-07T03:51:03.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108900 Milliseconds Vbucket 199 2016-04-07T03:51:03.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108930 Milliseconds Vbucket 199 2016-04-07T03:51:03.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108960 Milliseconds Vbucket 199 2016-04-07T03:51:03.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 108990 Milliseconds Vbucket 199 2016-04-07T03:51:03.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109020 Milliseconds Vbucket 199 2016-04-07T03:51:03.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109050 Milliseconds Vbucket 199 2016-04-07T03:51:03.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109080 Milliseconds Vbucket 199 2016-04-07T03:51:03.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109110 Milliseconds Vbucket 199 2016-04-07T03:51:03.675-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.751µs CommitTime 509.871298ms TotalFlushTime 10m23.433590283s TotalCommitTime 43.283415779s 2016-04-07T03:51:03.677-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 15383028, 15383028, 116 committed:true 2016-04-07T03:51:03.677-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 15383028, 15383028, 116 committed:true) SnapCreateDur 522.123439ms SnapOpenDur 60.717µs 2016-04-07T03:51:03.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109140 Milliseconds Vbucket 199 2016-04-07T03:51:03.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109170 Milliseconds Vbucket 199 2016-04-07T03:51:03.715-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:03.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109200 Milliseconds Vbucket 199 2016-04-07T03:51:03.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109230 Milliseconds Vbucket 199 2016-04-07T03:51:03.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109260 Milliseconds Vbucket 199 2016-04-07T03:51:03.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109290 Milliseconds Vbucket 199 2016-04-07T03:51:03.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109320 Milliseconds Vbucket 199 2016-04-07T03:51:03.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109350 Milliseconds Vbucket 199 2016-04-07T03:51:03.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109380 Milliseconds Vbucket 199 2016-04-07T03:51:03.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109410 Milliseconds Vbucket 199 2016-04-07T03:51:03.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109440 Milliseconds Vbucket 199 2016-04-07T03:51:04.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109470 Milliseconds Vbucket 199 2016-04-07T03:51:04.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109500 Milliseconds Vbucket 199 2016-04-07T03:51:04.053-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15390000 QueuedCount 50000 2016-04-07T03:51:04.067-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:04.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109530 Milliseconds Vbucket 199 2016-04-07T03:51:04.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109560 Milliseconds Vbucket 199 2016-04-07T03:51:04.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109590 Milliseconds Vbucket 199 2016-04-07T03:51:04.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109620 Milliseconds Vbucket 199 2016-04-07T03:51:04.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109650 Milliseconds Vbucket 199 2016-04-07T03:51:04.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109680 Milliseconds Vbucket 199 2016-04-07T03:51:04.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109710 Milliseconds Vbucket 199 2016-04-07T03:51:04.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109740 Milliseconds Vbucket 199 2016-04-07T03:51:04.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109770 Milliseconds Vbucket 199 2016-04-07T03:51:04.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109800 Milliseconds Vbucket 199 2016-04-07T03:51:04.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109830 Milliseconds Vbucket 199 2016-04-07T03:51:04.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109860 Milliseconds Vbucket 199 2016-04-07T03:51:04.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109890 Milliseconds Vbucket 199 2016-04-07T03:51:04.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109920 Milliseconds Vbucket 199 2016-04-07T03:51:04.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109950 Milliseconds Vbucket 199 2016-04-07T03:51:04.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 109980 Milliseconds Vbucket 199 2016-04-07T03:51:04.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110010 Milliseconds Vbucket 199 2016-04-07T03:51:04.576-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15400000 QueuedCount 50000 2016-04-07T03:51:04.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110040 Milliseconds Vbucket 199 2016-04-07T03:51:04.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110070 Milliseconds Vbucket 199 2016-04-07T03:51:04.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110100 Milliseconds Vbucket 199 2016-04-07T03:51:04.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110130 Milliseconds Vbucket 199 2016-04-07T03:51:04.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110160 Milliseconds Vbucket 199 2016-04-07T03:51:04.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110190 Milliseconds Vbucket 199 2016-04-07T03:51:04.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110220 Milliseconds Vbucket 199 2016-04-07T03:51:04.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110250 Milliseconds Vbucket 199 2016-04-07T03:51:04.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110280 Milliseconds Vbucket 199 2016-04-07T03:51:04.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110310 Milliseconds Vbucket 199 2016-04-07T03:51:04.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110340 Milliseconds Vbucket 199 2016-04-07T03:51:04.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110370 Milliseconds Vbucket 199 2016-04-07T03:51:04.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110400 Milliseconds Vbucket 199 2016-04-07T03:51:04.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110430 Milliseconds Vbucket 199 2016-04-07T03:51:05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110460 Milliseconds Vbucket 199 2016-04-07T03:51:05.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110490 Milliseconds Vbucket 199 2016-04-07T03:51:05.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110520 Milliseconds Vbucket 199 2016-04-07T03:51:05.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110550 Milliseconds Vbucket 199 2016-04-07T03:51:05.105-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15410000 QueuedCount 50000 2016-04-07T03:51:05.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110580 Milliseconds Vbucket 199 2016-04-07T03:51:05.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110610 Milliseconds Vbucket 199 2016-04-07T03:51:05.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110640 Milliseconds Vbucket 199 2016-04-07T03:51:05.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110670 Milliseconds Vbucket 199 2016-04-07T03:51:05.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110700 Milliseconds Vbucket 199 2016-04-07T03:51:05.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110730 Milliseconds Vbucket 199 2016-04-07T03:51:05.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110760 Milliseconds Vbucket 199 2016-04-07T03:51:05.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110790 Milliseconds Vbucket 199 2016-04-07T03:51:05.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110820 Milliseconds Vbucket 199 2016-04-07T03:51:05.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110850 Milliseconds Vbucket 199 2016-04-07T03:51:05.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110880 Milliseconds Vbucket 199 2016-04-07T03:51:05.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110910 Milliseconds Vbucket 199 2016-04-07T03:51:05.465-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:05.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 110940 Milliseconds Vbucket 199 2016-04-07T03:51:05.536-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15420000 QueuedCount 50000 2016-04-07T03:51:05.923-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15430000 QueuedCount 50000 2016-04-07T03:51:06.379-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15440000 QueuedCount 50000 2016-04-07T03:51:06.785-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15450000 QueuedCount 48676 2016-04-07T03:51:07.164-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15460000 QueuedCount 38676 2016-04-07T03:51:07.559-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15470000 QueuedCount 28676 2016-04-07T03:51:07.959-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15480000 QueuedCount 18676 2016-04-07T03:51:08.318-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15490000 QueuedCount 8676 2016-04-07T03:51:09.134-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.649µs CommitTime 513.882281ms TotalFlushTime 10m28.168774958s TotalCommitTime 43.79729806s 2016-04-07T03:51:09.134-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 15498676, 15498676, 117 committed:true 2016-04-07T03:51:09.135-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 15498676, 15498676, 117 committed:true) SnapCreateDur 523.907892ms SnapOpenDur 48.103µs 2016-04-07T03:51:09.171-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20000000 2016-04-07T03:51:09.175-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:09.177-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15500000 QueuedCount 27038 2016-04-07T03:51:09.46-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:09.652-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15510000 QueuedCount 50000 2016-04-07T03:51:10.125-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15520000 QueuedCount 50000 2016-04-07T03:51:10.621-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15530000 QueuedCount 50000 2016-04-07T03:51:10.714-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:11.058-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15540000 QueuedCount 50000 2016-04-07T03:51:11.518-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15550000 QueuedCount 50000 2016-04-07T03:51:11.939-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15560000 QueuedCount 49324 2016-04-07T03:51:12.33-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15570000 QueuedCount 39324 2016-04-07T03:51:12.685-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15580000 QueuedCount 29324 2016-04-07T03:51:12.944-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":4390676,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":6095898556,"bucket-1:myindex:avg_ts_items_count":111478,"bucket-1:myindex:build_progress":77,"bucket-1:myindex:data_size":5035383598,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":30340841472,"bucket-1:myindex:flush_queue_size":22563,"bucket-1:myindex:frag_percent":83,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":1632092972,"bucket-1:myindex:items_count":15498676,"bucket-1:myindex:num_commits":117,"bucket-1:myindex:num_compactions":2,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":15582467,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":4417533,"bucket-1:myindex:num_flush_queued":15609324,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":117,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":5457939520,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"117 43797298060 -9080951141138076","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"15586762 100123786949 610883243687809041","bucket-1:myindex:timings/storage_handle_open":"117 2816190 70691551916","bucket-1:myindex:timings/storage_info":"469 4429739 113569856319","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"118 76045890 732094249091826","bucket-1:myindex:timings/storage_meta_set":"117 374279357 107833870300171135","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"31173522 521535801078 2486004303477189200","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":415,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"2626 332299625 56420867292485"} 2016-04-07T03:51:13.066-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15590000 QueuedCount 19324 2016-04-07T03:51:13.481-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15600000 QueuedCount 9324 2016-04-07T03:51:14.335-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.231µs CommitTime 473.615075ms TotalFlushTime 10m32.70034141s TotalCommitTime 44.270913135s 2016-04-07T03:51:14.335-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 15609324, 15609324, 118 committed:true 2016-04-07T03:51:14.335-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 15609324, 15609324, 118 committed:true) SnapCreateDur 483.515593ms SnapOpenDur 50.845µs 2016-04-07T03:51:14.37-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15610000 QueuedCount 23401 2016-04-07T03:51:14.373-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:14.559-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:14.834-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15620000 QueuedCount 50000 2016-04-07T03:51:15.283-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15630000 QueuedCount 50000 2016-04-07T03:51:15.67-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15640000 QueuedCount 50000 2016-04-07T03:51:15.738-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:16.123-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15650000 QueuedCount 50000 2016-04-07T03:51:17.468-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15660000 QueuedCount 50000 2016-04-07T03:51:17.935-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15670000 QueuedCount 45035 2016-04-07T03:51:18.337-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15680000 QueuedCount 35035 2016-04-07T03:51:18.706-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15690000 QueuedCount 25035 2016-04-07T03:51:19.059-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15700000 QueuedCount 15035 2016-04-07T03:51:19.433-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15710000 QueuedCount 5035 2016-04-07T03:51:20.163-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.715µs CommitTime 546.849327ms TotalFlushTime 10m37.801482874s TotalCommitTime 44.817762462s 2016-04-07T03:51:20.163-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 15715035, 15715035, 119 committed:true 2016-04-07T03:51:20.163-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 15715035, 15715035, 119 committed:true) SnapCreateDur 564.632853ms SnapOpenDur 39.377µs 2016-04-07T03:51:20.203-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:20.239-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:20.408-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15720000 QueuedCount 50000 2016-04-07T03:51:20.901-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15730000 QueuedCount 50000 2016-04-07T03:51:21.356-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15740000 QueuedCount 50000 2016-04-07T03:51:21.378-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:21.787-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15750000 QueuedCount 50000 2016-04-07T03:51:22.201-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15760000 QueuedCount 50000 2016-04-07T03:51:22.61-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15770000 QueuedCount 46801 2016-04-07T03:51:22.99-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15780000 QueuedCount 36801 2016-04-07T03:51:23.421-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15790000 QueuedCount 26801 2016-04-07T03:51:23.798-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15800000 QueuedCount 16801 2016-04-07T03:51:24.174-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15810000 QueuedCount 6801 2016-04-07T03:51:24.952-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.37µs CommitTime 514.378717ms TotalFlushTime 10m41.900460353s TotalCommitTime 45.332141179s 2016-04-07T03:51:24.952-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 15816801, 15816801, 120 committed:true 2016-04-07T03:51:24.952-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 15816801, 15816801, 120 committed:true) SnapCreateDur 525.381596ms SnapOpenDur 36.551µs 2016-04-07T03:51:24.991-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:25.104-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15820000 QueuedCount 50000 2016-04-07T03:51:25.608-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15830000 QueuedCount 50000 2016-04-07T03:51:26.003-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:26.074-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15840000 QueuedCount 50000 2016-04-07T03:51:26.579-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15850000 QueuedCount 50000 2016-04-07T03:51:27.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15860000 QueuedCount 50000 2016-04-07T03:51:27.454-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:27.477-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15870000 QueuedCount 50000 2016-04-07T03:51:27.966-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15880000 QueuedCount 50000 2016-04-07T03:51:28.449-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15890000 QueuedCount 50000 2016-04-07T03:51:28.926-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15900000 QueuedCount 47953 2016-04-07T03:51:29.3-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15910000 QueuedCount 37953 2016-04-07T03:51:29.692-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15920000 QueuedCount 27953 2016-04-07T03:51:30.087-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15930000 QueuedCount 17953 2016-04-07T03:51:30.462-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15940000 QueuedCount 7953 2016-04-07T03:51:31.346-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.691µs CommitTime 558.607921ms TotalFlushTime 10m47.494880707s TotalCommitTime 45.8907491s 2016-04-07T03:51:31.346-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 15947953, 15947953, 121 committed:true 2016-04-07T03:51:31.346-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 15947953, 15947953, 121 committed:true) SnapCreateDur 569.621153ms SnapOpenDur 43.433µs 2016-04-07T03:51:31.384-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:31.398-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15950000 QueuedCount 25960 2016-04-07T03:51:31.7-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:31.919-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15960000 QueuedCount 50000 2016-04-07T03:51:32.397-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15970000 QueuedCount 50000 2016-04-07T03:51:32.845-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15980000 QueuedCount 50000 2016-04-07T03:51:33.096-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:33.266-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 15990000 QueuedCount 50000 2016-04-07T03:51:33.793-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16000000 QueuedCount 50000 2016-04-07T03:51:36.147-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16010000 QueuedCount 50000 2016-04-07T03:51:36.568-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16020000 QueuedCount 45362 2016-04-07T03:51:36.946-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16030000 QueuedCount 35362 2016-04-07T03:51:37.258-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16040000 QueuedCount 25362 2016-04-07T03:51:37.652-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16050000 QueuedCount 15362 2016-04-07T03:51:38.069-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16060000 QueuedCount 5362 2016-04-07T03:51:39.064-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.687µs CommitTime 742.517424ms TotalFlushTime 10m54.269959145s TotalCommitTime 46.633266524s 2016-04-07T03:51:39.064-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 16065362, 16065362, 122 committed:true 2016-04-07T03:51:39.065-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 16065362, 16065362, 122 committed:true) SnapCreateDur 752.444364ms SnapOpenDur 81.135µs 2016-04-07T03:51:39.32-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16070000 QueuedCount 24352 2016-04-07T03:51:39.842-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16080000 QueuedCount 14352 2016-04-07T03:51:40.315-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16090000 QueuedCount 4352 2016-04-07T03:51:40.868-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.384µs CommitTime 355.112187ms TotalFlushTime 10m55.670030664s TotalCommitTime 46.988378711s 2016-04-07T03:51:40.868-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 16094352, 16094352, 123 committed:true 2016-04-07T03:51:40.868-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 16094352, 16094352, 123 committed:true) SnapCreateDur 365.651241ms SnapOpenDur 55.972µs 2016-04-07T03:51:40.906-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:41.136-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16100000 QueuedCount 50000 2016-04-07T03:51:41.341-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:41.645-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16110000 QueuedCount 50000 2016-04-07T03:51:42.129-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16120000 QueuedCount 50000 2016-04-07T03:51:42.616-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16130000 QueuedCount 50000 2016-04-07T03:51:42.716-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:43.171-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16140000 QueuedCount 50000 2016-04-07T03:51:43.687-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16150000 QueuedCount 50000 2016-04-07T03:51:44.109-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16160000 QueuedCount 50000 2016-04-07T03:51:44.121-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:44.541-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16170000 QueuedCount 49999 2016-04-07T03:51:45.02-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16180000 QueuedCount 50000 2016-04-07T03:51:45.443-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16190000 QueuedCount 49877 2016-04-07T03:51:45.815-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16200000 QueuedCount 39877 2016-04-07T03:51:46.171-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16210000 QueuedCount 29877 2016-04-07T03:51:46.53-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16220000 QueuedCount 19877 2016-04-07T03:51:46.901-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16230000 QueuedCount 9877 2016-04-07T03:51:47.781-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.652µs CommitTime 509.376353ms TotalFlushTime 11m1.795815806s TotalCommitTime 47.497755064s 2016-04-07T03:51:47.782-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 16239877, 16239877, 124 committed:true 2016-04-07T03:51:47.782-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 16239877, 16239877, 124 committed:true) SnapCreateDur 519.250889ms SnapOpenDur 51.72µs 2016-04-07T03:51:47.805-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16240000 QueuedCount 4750 2016-04-07T03:51:47.822-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:48.049-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:48.234-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16250000 QueuedCount 50000 2016-04-07T03:51:48.721-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16260000 QueuedCount 50000 2016-04-07T03:51:49.17-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16270000 QueuedCount 50000 2016-04-07T03:51:49.322-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:49.579-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16280000 QueuedCount 50000 2016-04-07T03:51:50.071-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16290000 QueuedCount 50000 2016-04-07T03:51:50.555-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16300000 QueuedCount 50000 2016-04-07T03:51:50.978-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16310000 QueuedCount 41207 2016-04-07T03:51:51.407-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16320000 QueuedCount 31207 2016-04-07T03:51:51.757-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16330000 QueuedCount 21207 2016-04-07T03:51:54.053-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16340000 QueuedCount 11207 2016-04-07T03:51:54.411-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16350000 QueuedCount 1207 2016-04-07T03:51:54.979-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.347µs CommitTime 491.31292ms TotalFlushTime 11m8.312405381s TotalCommitTime 47.989067984s 2016-04-07T03:51:54.979-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 16351207, 16351207, 125 committed:true 2016-04-07T03:51:54.979-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 16351207, 16351207, 125 committed:true) SnapCreateDur 501.709135ms SnapOpenDur 48.531µs 2016-04-07T03:51:55.377-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16360000 QueuedCount 15783 2016-04-07T03:51:55.775-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16370000 QueuedCount 5783 2016-04-07T03:51:56.266-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.899µs CommitTime 270.270783ms TotalFlushTime 11m9.278655157s TotalCommitTime 48.259338767s 2016-04-07T03:51:56.266-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 16375783, 16375783, 126 committed:true 2016-04-07T03:51:56.266-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 16375783, 16375783, 126 committed:true) SnapCreateDur 281.596914ms SnapOpenDur 43.482µs 2016-04-07T03:51:56.299-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:56.333-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:56.901-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16380000 QueuedCount 50000 2016-04-07T03:51:57.315-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16390000 QueuedCount 50000 2016-04-07T03:51:57.735-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:57.844-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16400000 QueuedCount 50000 2016-04-07T03:51:58.318-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16410000 QueuedCount 50000 2016-04-07T03:51:58.783-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16420000 QueuedCount 50000 2016-04-07T03:51:58.872-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:51:59.231-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16430000 QueuedCount 50000 2016-04-07T03:51:59.692-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16440000 QueuedCount 50000 2016-04-07T03:51:59.99-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:00.154-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16450000 QueuedCount 50000 2016-04-07T03:52:00.585-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16460000 QueuedCount 50000 2016-04-07T03:52:00.948-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16470000 QueuedCount 50000 2016-04-07T03:52:01.388-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16480000 QueuedCount 42662 2016-04-07T03:52:01.835-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16490000 QueuedCount 32662 2016-04-07T03:52:02.308-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16500000 QueuedCount 22662 2016-04-07T03:52:02.787-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16510000 QueuedCount 12662 2016-04-07T03:52:03.225-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16520000 QueuedCount 2662 2016-04-07T03:52:03.921-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.593µs CommitTime 582.590149ms TotalFlushTime 11m16.065504401s TotalCommitTime 48.841928916s 2016-04-07T03:52:03.921-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 16522662, 16522662, 127 committed:true 2016-04-07T03:52:03.921-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 16522662, 16522662, 127 committed:true) SnapCreateDur 593.935227ms SnapOpenDur 47.301µs 2016-04-07T03:52:03.957-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:09.426-07:00 [Info] CompactionDaemon: Finished compacting index instance:9366861204008862690 2016-04-07T03:52:09.437-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:09.706-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16530000 QueuedCount 50000 2016-04-07T03:52:10.135-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16540000 QueuedCount 45378 2016-04-07T03:52:10.522-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16550000 QueuedCount 35378 2016-04-07T03:52:10.884-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16560000 QueuedCount 25378 2016-04-07T03:52:11.238-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16570000 QueuedCount 15378 2016-04-07T03:52:11.577-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16580000 QueuedCount 5378 2016-04-07T03:52:12.246-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.274µs CommitTime 430.443574ms TotalFlushTime 11m23.874235198s TotalCommitTime 49.27237249s 2016-04-07T03:52:12.246-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 16585378, 16585378, 128 committed:true 2016-04-07T03:52:12.246-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 16585378, 16585378, 128 committed:true) SnapCreateDur 441.53506ms SnapOpenDur 57.67µs 2016-04-07T03:52:12.269-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:12.316-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:12.339-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:12.477-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16590000 QueuedCount 50000 2016-04-07T03:52:12.707-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:12.802-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:12.944-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":3350282,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":7036831797,"bucket-1:myindex:avg_ts_items_count":84948,"bucket-1:myindex:build_progress":82,"bucket-1:myindex:data_size":5361697415,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":30827536384,"bucket-1:myindex:flush_queue_size":50013,"bucket-1:myindex:frag_percent":82,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":1740471872,"bucket-1:myindex:items_count":16522662,"bucket-1:myindex:num_commits":128,"bucket-1:myindex:num_compactions":2,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":16573328,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":3426672,"bucket-1:myindex:num_flush_queued":16649730,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":128,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":8324407459,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"128 49272372490 2866931441380173328","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"16599718 106856894980 610932131262222036","bucket-1:myindex:timings/storage_handle_open":"128 3113917 79271770167","bucket-1:myindex:timings/storage_info":"513 5003091 134189684335","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"129 79763087 733365031967961","bucket-1:myindex:timings/storage_meta_set":"128 378883832 107835821367683752","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"33199434 566837646409 -2440428564633923731","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":373,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"2769 356858904 60739614845486"} 2016-04-07T03:52:12.966-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16600000 QueuedCount 50000 2016-04-07T03:52:12.973-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:13.063-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:13.162-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:13.239-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:13.401-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16610000 QueuedCount 50000 2016-04-07T03:52:13.402-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:13.492-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:13.666-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:13.767-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16620000 QueuedCount 50000 2016-04-07T03:52:13.909-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:13.93-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:14.179-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16630000 QueuedCount 50000 2016-04-07T03:52:14.503-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:14.583-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16640000 QueuedCount 50000 2016-04-07T03:52:15.049-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16650000 QueuedCount 50000 2016-04-07T03:52:15.476-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16660000 QueuedCount 50000 2016-04-07T03:52:15.89-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16670000 QueuedCount 50000 2016-04-07T03:52:16.307-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16680000 QueuedCount 50000 2016-04-07T03:52:16.711-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16690000 QueuedCount 41007 2016-04-07T03:52:17.04-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16700000 QueuedCount 31007 2016-04-07T03:52:17.422-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16710000 QueuedCount 21007 2016-04-07T03:52:17.781-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16720000 QueuedCount 11007 2016-04-07T03:52:18.137-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16730000 QueuedCount 1007 2016-04-07T03:52:18.738-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.132µs CommitTime 527.007275ms TotalFlushTime 11m29.596674192s TotalCommitTime 49.799379765s 2016-04-07T03:52:18.739-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 16731007, 16731007, 129 committed:true 2016-04-07T03:52:18.739-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 16731007, 16731007, 129 committed:true) SnapCreateDur 536.887948ms SnapOpenDur 45.097µs 2016-04-07T03:52:18.775-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:18.801-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:18.825-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:18.847-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:18.87-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:18.894-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:19.042-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:19.134-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16740000 QueuedCount 50000 2016-04-07T03:52:19.292-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:19.549-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16750000 QueuedCount 50000 2016-04-07T03:52:19.867-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:19.941-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16760000 QueuedCount 48149 2016-04-07T03:52:20.29-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16770000 QueuedCount 38149 2016-04-07T03:52:20.592-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16780000 QueuedCount 28149 2016-04-07T03:52:20.947-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16790000 QueuedCount 18149 2016-04-07T03:52:21.288-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16800000 QueuedCount 8149 2016-04-07T03:52:22.081-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.173µs CommitTime 483.382302ms TotalFlushTime 11m32.350674517s TotalCommitTime 50.282762067s 2016-04-07T03:52:22.081-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 16808149, 16808149, 130 committed:true 2016-04-07T03:52:22.081-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 16808149, 16808149, 130 committed:true) SnapCreateDur 494.423332ms SnapOpenDur 32.173µs 2016-04-07T03:52:22.102-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.123-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.143-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.164-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.184-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.205-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.229-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.23-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16810000 QueuedCount 5386 2016-04-07T03:52:22.398-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.423-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.449-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.473-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.615-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:22.793-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16820000 QueuedCount 50000 2016-04-07T03:52:23.236-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16830000 QueuedCount 50000 2016-04-07T03:52:23.662-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16840000 QueuedCount 50000 2016-04-07T03:52:23.739-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:23.76-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:23.91-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:23.931-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:23.951-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:23.972-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:24.097-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:24.103-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16850000 QueuedCount 49430 2016-04-07T03:52:24.188-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:24.446-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:24.527-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16860000 QueuedCount 50000 2016-04-07T03:52:24.661-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:24.911-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:24.982-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16870000 QueuedCount 50000 2016-04-07T03:52:25.35-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16880000 QueuedCount 43387 2016-04-07T03:52:25.653-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16890000 QueuedCount 33387 2016-04-07T03:52:26.009-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16900000 QueuedCount 23387 2016-04-07T03:52:26.361-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16910000 QueuedCount 13387 2016-04-07T03:52:26.719-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16920000 QueuedCount 3387 2016-04-07T03:52:27.376-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.928µs CommitTime 503.454179ms TotalFlushTime 11m36.869344425s TotalCommitTime 50.786216246s 2016-04-07T03:52:27.376-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 16923387, 16923387, 131 committed:true 2016-04-07T03:52:27.377-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 16923387, 16923387, 131 committed:true) SnapCreateDur 514.008078ms SnapOpenDur 44.133µs 2016-04-07T03:52:27.407-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:27.448-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:27.684-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16930000 QueuedCount 50000 2016-04-07T03:52:27.878-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:28.122-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16940000 QueuedCount 50000 2016-04-07T03:52:28.203-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:28.224-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:28.364-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:28.478-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16950000 QueuedCount 50000 2016-04-07T03:52:28.537-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:28.626-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:28.8-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:28.876-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16960000 QueuedCount 50000 2016-04-07T03:52:28.957-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:29.205-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:29.282-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:29.29-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16970000 QueuedCount 49233 2016-04-07T03:52:29.449-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:29.593-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:29.688-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16980000 QueuedCount 50000 2016-04-07T03:52:29.768-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:29.93-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:30.096-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 16990000 QueuedCount 50000 2016-04-07T03:52:30.486-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17000000 QueuedCount 43917 2016-04-07T03:52:30.864-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17010000 QueuedCount 33917 2016-04-07T03:52:31.225-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17020000 QueuedCount 23917 2016-04-07T03:52:31.601-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17030000 QueuedCount 13917 2016-04-07T03:52:31.933-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17040000 QueuedCount 3917 2016-04-07T03:52:32.675-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.651µs CommitTime 579.343604ms TotalFlushTime 11m41.409709425s TotalCommitTime 51.36555985s 2016-04-07T03:52:32.675-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 17043917, 17043917, 132 committed:true 2016-04-07T03:52:32.675-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 17043917, 17043917, 132 committed:true) SnapCreateDur 590.29486ms SnapOpenDur 52.853µs 2016-04-07T03:52:32.698-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:32.737-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:32.914-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17050000 QueuedCount 50000 2016-04-07T03:52:33.346-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17060000 QueuedCount 50000 2016-04-07T03:52:33.663-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:33.759-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17070000 QueuedCount 50000 2016-04-07T03:52:33.91-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:33.931-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:34.003-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:34.095-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:34.165-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17080000 QueuedCount 50000 2016-04-07T03:52:34.331-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:34.502-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:34.569-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17090000 QueuedCount 50000 2016-04-07T03:52:34.713-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:34.955-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:34.96-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17100000 QueuedCount 49502 2016-04-07T03:52:35.327-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17110000 QueuedCount 45298 2016-04-07T03:52:35.671-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17120000 QueuedCount 35298 2016-04-07T03:52:36.043-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17130000 QueuedCount 25298 2016-04-07T03:52:36.351-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17140000 QueuedCount 15298 2016-04-07T03:52:36.7-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17150000 QueuedCount 5298 2016-04-07T03:52:37.632-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.847µs CommitTime 711.819577ms TotalFlushTime 11m45.489463218s TotalCommitTime 52.077379427s 2016-04-07T03:52:37.632-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 17155298, 17155298, 133 committed:true 2016-04-07T03:52:37.632-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 17155298, 17155298, 133 committed:true) SnapCreateDur 721.425397ms SnapOpenDur 37.793µs 2016-04-07T03:52:37.656-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:37.678-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:37.702-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:37.726-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:37.751-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:37.78-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:37.803-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:37.827-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:37.849-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:37.861-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17160000 QueuedCount 44192 2016-04-07T03:52:37.874-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:38.237-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:38.328-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17170000 QueuedCount 50000 2016-04-07T03:52:38.607-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:38.774-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17180000 QueuedCount 50000 2016-04-07T03:52:38.864-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:38.886-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:39.062-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:39.165-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17190000 QueuedCount 50000 2016-04-07T03:52:39.245-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:39.415-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:39.427-07:00 [Info] CompactionDaemon: Checking fragmentation, IndexInstId: 9366861204008862690 Data:5563349688, Disk:32952807424, ExtraSnapshotData:0, Fragmentation:83.11722088981064% 2016-04-07T03:52:39.427-07:00 [Info] CompactionDaemon: Compacting index instance:9366861204008862690 2016-04-07T03:52:39.489-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:39.491-07:00 [Info] ForestDBSlice::Compact Compacting upto SeqNum 16731007. Slice Id 0, IndexInstId 9366861204008862690, IndexDefnId 9366861204008862690 2016-04-07T03:52:40.088-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17200000 QueuedCount 50000 2016-04-07T03:52:40.597-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17210000 QueuedCount 50000 2016-04-07T03:52:41.072-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17220000 QueuedCount 50000 2016-04-07T03:52:41.532-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17230000 QueuedCount 49988 2016-04-07T03:52:41.742-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:41.958-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17240000 QueuedCount 50000 2016-04-07T03:52:42.394-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17250000 QueuedCount 41266 2016-04-07T03:52:42.828-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17260000 QueuedCount 31266 2016-04-07T03:52:43.255-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17270000 QueuedCount 21266 2016-04-07T03:52:43.662-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17280000 QueuedCount 11266 2016-04-07T03:52:44.087-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17290000 QueuedCount 1266 2016-04-07T03:52:44.68-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.41µs CommitTime 557.68028ms TotalFlushTime 11m51.743520917s TotalCommitTime 52.635059707s 2016-04-07T03:52:44.68-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 17291266, 17291266, 134 committed:true 2016-04-07T03:52:44.68-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 17291266, 17291266, 134 committed:true) SnapCreateDur 569.515385ms SnapOpenDur 37.835µs 2016-04-07T03:52:44.7-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.721-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.742-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.762-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.783-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.804-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.826-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.847-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.869-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.89-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.913-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.936-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.96-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:44.984-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:45.008-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:45.032-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:45.058-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:45.203-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:45.208-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17300000 QueuedCount 49566 2016-04-07T03:52:45.224-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:45.706-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17310000 QueuedCount 50000 2016-04-07T03:52:46.111-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:46.19-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17320000 QueuedCount 50000 2016-04-07T03:52:46.617-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17330000 QueuedCount 50000 2016-04-07T03:52:47.075-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17340000 QueuedCount 50000 2016-04-07T03:52:47.534-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17350000 QueuedCount 50000 2016-04-07T03:52:48.01-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17360000 QueuedCount 50000 2016-04-07T03:52:48.109-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:48.447-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17370000 QueuedCount 46345 2016-04-07T03:52:48.859-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17380000 QueuedCount 36345 2016-04-07T03:52:49.293-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17390000 QueuedCount 26345 2016-04-07T03:52:49.707-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17400000 QueuedCount 16345 2016-04-07T03:52:50.079-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17410000 QueuedCount 6345 2016-04-07T03:52:50.976-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.838µs CommitTime 570.383821ms TotalFlushTime 11m57.169844541s TotalCommitTime 53.205443528s 2016-04-07T03:52:50.976-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 17416345, 17416345, 135 committed:true 2016-04-07T03:52:50.976-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 17416345, 17416345, 135 committed:true) SnapCreateDur 580.558625ms SnapOpenDur 66.147µs 2016-04-07T03:52:51.002-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:51.024-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:51.046-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:51.069-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:51.142-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17420000 QueuedCount 49991 2016-04-07T03:52:51.624-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17430000 QueuedCount 50000 2016-04-07T03:52:51.718-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:52.098-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17440000 QueuedCount 50000 2016-04-07T03:52:52.555-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17450000 QueuedCount 50000 2016-04-07T03:52:52.99-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17460000 QueuedCount 50000 2016-04-07T03:52:53.007-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:53.169-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:53.413-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17470000 QueuedCount 50000 2016-04-07T03:52:53.847-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17480000 QueuedCount 44627 2016-04-07T03:52:54.188-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17490000 QueuedCount 34627 2016-04-07T03:52:54.569-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17500000 QueuedCount 24627 2016-04-07T03:52:54.953-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17510000 QueuedCount 14627 2016-04-07T03:52:55.334-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17520000 QueuedCount 4627 2016-04-07T03:52:56.064-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.681µs CommitTime 553.333479ms TotalFlushTime 12m1.518015009s TotalCommitTime 53.758777007s 2016-04-07T03:52:56.065-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 17524627, 17524627, 136 committed:true 2016-04-07T03:52:56.065-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 17524627, 17524627, 136 committed:true) SnapCreateDur 563.758923ms SnapOpenDur 65.028µs 2016-04-07T03:52:56.086-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.11-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.136-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.157-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.177-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.198-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.219-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.239-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.26-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.281-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.301-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17530000 QueuedCount 9455 2016-04-07T03:52:56.302-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.326-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.35-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.375-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.397-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.723-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:56.798-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17540000 QueuedCount 50000 2016-04-07T03:52:56.799-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:57.226-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17550000 QueuedCount 50000 2016-04-07T03:52:57.675-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17560000 QueuedCount 50000 2016-04-07T03:52:57.763-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:58.116-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17570000 QueuedCount 50000 2016-04-07T03:52:58.557-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17580000 QueuedCount 50000 2016-04-07T03:52:58.642-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:52:58.912-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17590000 QueuedCount 45743 2016-04-07T03:52:59.3-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17600000 QueuedCount 35743 2016-04-07T03:52:59.693-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17610000 QueuedCount 25743 2016-04-07T03:53:00.119-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17620000 QueuedCount 15743 2016-04-07T03:53:00.504-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17630000 QueuedCount 5743 2016-04-07T03:53:01.292-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.384µs CommitTime 543.450967ms TotalFlushTime 12m6.021207958s TotalCommitTime 54.302227974s 2016-04-07T03:53:01.292-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 17635743, 17635743, 137 committed:true 2016-04-07T03:53:01.292-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 17635743, 17635743, 137 committed:true) SnapCreateDur 554.437701ms SnapOpenDur 43.586µs 2016-04-07T03:53:01.319-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:01.346-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:01.366-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:01.397-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:01.422-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:01.444-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:01.468-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.092-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.112-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.133-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.179-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17640000 QueuedCount 50000 2016-04-07T03:53:02.18-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.201-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.221-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.243-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.264-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.285-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.306-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.566-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.583-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17650000 QueuedCount 48490 2016-04-07T03:53:02.834-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.855-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:02.948-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:03.036-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17660000 QueuedCount 50000 2016-04-07T03:53:03.208-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:03.296-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:03.481-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17670000 QueuedCount 50000 2016-04-07T03:53:03.485-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:03.665-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:03.92-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17680000 QueuedCount 49731 2016-04-07T03:53:04.339-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17690000 QueuedCount 39731 2016-04-07T03:53:04.726-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17700000 QueuedCount 29731 2016-04-07T03:53:05.119-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17710000 QueuedCount 19731 2016-04-07T03:53:05.522-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17720000 QueuedCount 9731 2016-04-07T03:53:06.469-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.774µs CommitTime 582.546228ms TotalFlushTime 12m10.477261936s TotalCommitTime 54.884774202s 2016-04-07T03:53:06.47-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 17729731, 17729731, 138 committed:true 2016-04-07T03:53:06.47-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 17729731, 17729731, 138 committed:true) SnapCreateDur 592.497184ms SnapOpenDur 64.649µs 2016-04-07T03:53:06.495-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17730000 QueuedCount 7117 2016-04-07T03:53:06.495-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:06.528-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:06.596-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:06.979-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17740000 QueuedCount 50000 2016-04-07T03:53:07.379-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17750000 QueuedCount 50000 2016-04-07T03:53:07.651-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:07.831-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17760000 QueuedCount 50000 2016-04-07T03:53:08.009-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:08.16-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:08.265-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17770000 QueuedCount 50000 2016-04-07T03:53:08.458-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:08.71-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17780000 QueuedCount 50000 2016-04-07T03:53:08.888-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:09.154-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17790000 QueuedCount 50000 2016-04-07T03:53:10.23-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17800000 QueuedCount 40591 2016-04-07T03:53:10.606-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17810000 QueuedCount 30591 2016-04-07T03:53:10.993-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17820000 QueuedCount 20591 2016-04-07T03:53:11.324-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17830000 QueuedCount 10591 2016-04-07T03:53:11.686-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17840000 QueuedCount 591 2016-04-07T03:53:12.326-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.036µs CommitTime 563.096112ms TotalFlushTime 12m15.585490306s TotalCommitTime 55.447870314s 2016-04-07T03:53:12.326-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 17840591, 17840591, 139 committed:true 2016-04-07T03:53:12.326-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 17840591, 17840591, 139 committed:true) SnapCreateDur 574.061618ms SnapOpenDur 50.602µs 2016-04-07T03:53:12.347-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:12.368-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:12.394-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:12.416-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:12.454-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:12.482-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:12.785-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17850000 QueuedCount 50000 2016-04-07T03:53:12.881-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:12.944-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":2097118,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":5580616958,"bucket-1:myindex:avg_ts_items_count":107371,"bucket-1:myindex:build_progress":89,"bucket-1:myindex:data_size":5770516114,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":35093520384,"bucket-1:myindex:flush_queue_size":50024,"bucket-1:myindex:frag_percent":83,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":1873021605,"bucket-1:myindex:items_count":17729731,"bucket-1:myindex:num_commits":139,"bucket-1:myindex:num_compactions":3,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":17833274,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":2166726,"bucket-1:myindex:num_flush_queued":17902905,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":139,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":5856429785,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"139 55447870314 6368418020800111442","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"17852882 114769821075 610985493548351063","bucket-1:myindex:timings/storage_handle_open":"139 3407338 87442232060","bucket-1:myindex:timings/storage_info":"557 5457599 145878569147","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"141 137293727 3662212490286249","bucket-1:myindex:timings/storage_meta_set":"139 382750315 107837198853450377","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"35705763 609738447162 -47110209729232180","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":187,"memory_quota":31457280000,"memory_used":4033445888,"needs_restart":false,"num_connections":0,"timings/stats_response":"2912 381763946 65181636415418"} 2016-04-07T03:53:12.971-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:13.138-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:13.239-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17860000 QueuedCount 50000 2016-04-07T03:53:13.509-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:13.606-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:13.689-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17870000 QueuedCount 50000 2016-04-07T03:53:13.875-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:14.127-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17880000 QueuedCount 50000 2016-04-07T03:53:14.576-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17890000 QueuedCount 50000 2016-04-07T03:53:14.812-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:15.034-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17900000 QueuedCount 46205 2016-04-07T03:53:15.382-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17910000 QueuedCount 36205 2016-04-07T03:53:15.784-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17920000 QueuedCount 26205 2016-04-07T03:53:16.177-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17930000 QueuedCount 16205 2016-04-07T03:53:16.561-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17940000 QueuedCount 6205 2016-04-07T03:53:17.612-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.661µs CommitTime 792.882536ms TotalFlushTime 12m19.898513903s TotalCommitTime 56.24075285s 2016-04-07T03:53:17.612-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 17946205, 17946205, 140 committed:true 2016-04-07T03:53:17.612-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 17946205, 17946205, 140 committed:true) SnapCreateDur 804.561738ms SnapOpenDur 38.852µs 2016-04-07T03:53:17.646-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:18.113-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17950000 QueuedCount 50000 2016-04-07T03:53:18.215-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:18.237-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:18.608-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17960000 QueuedCount 50000 2016-04-07T03:53:19.11-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17970000 QueuedCount 50000 2016-04-07T03:53:19.398-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:19.569-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17980000 QueuedCount 50000 2016-04-07T03:53:20.013-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 17990000 QueuedCount 50000 2016-04-07T03:53:20.445-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18000000 QueuedCount 50000 2016-04-07T03:53:20.74-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:20.923-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18010000 QueuedCount 50000 2016-04-07T03:53:21.355-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18020000 QueuedCount 50000 2016-04-07T03:53:21.732-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18030000 QueuedCount 50000 2016-04-07T03:53:22.272-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18040000 QueuedCount 50000 2016-04-07T03:53:22.691-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18050000 QueuedCount 40170 2016-04-07T03:53:23.116-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18060000 QueuedCount 30170 2016-04-07T03:53:23.525-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18070000 QueuedCount 20170 2016-04-07T03:53:23.896-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18080000 QueuedCount 10170 2016-04-07T03:53:24.274-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18090000 QueuedCount 170 2016-04-07T03:53:24.935-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.99µs CommitTime 640.987637ms TotalFlushTime 12m26.318913375s TotalCommitTime 56.881740487s 2016-04-07T03:53:24.935-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 18090170, 18090170, 141 committed:true 2016-04-07T03:53:24.935-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 18090170, 18090170, 141 committed:true) SnapCreateDur 650.792396ms SnapOpenDur 40.997µs 2016-04-07T03:53:24.957-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:24.987-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:25.02-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:25.042-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:25.062-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:25.087-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:25.373-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18100000 QueuedCount 50000 2016-04-07T03:53:25.835-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18110000 QueuedCount 50000 2016-04-07T03:53:26.294-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:26.306-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18120000 QueuedCount 48909 2016-04-07T03:53:26.771-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18130000 QueuedCount 49980 2016-04-07T03:53:27.202-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18140000 QueuedCount 39980 2016-04-07T03:53:27.587-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18150000 QueuedCount 29980 2016-04-07T03:53:27.971-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18160000 QueuedCount 19980 2016-04-07T03:53:28.313-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18170000 QueuedCount 9980 2016-04-07T03:53:29.22-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.407µs CommitTime 509.149991ms TotalFlushTime 12m29.956930494s TotalCommitTime 57.390890478s 2016-04-07T03:53:29.22-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 18179980, 18179980, 142 committed:true 2016-04-07T03:53:29.22-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 18179980, 18179980, 142 committed:true) SnapCreateDur 518.907781ms SnapOpenDur 36.333µs 2016-04-07T03:53:29.24-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18180000 QueuedCount 572 2016-04-07T03:53:29.269-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:29.29-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:29.311-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:29.336-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:30.46-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:30.55-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18190000 QueuedCount 50000 2016-04-07T03:53:31-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:31.009-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18200000 QueuedCount 49120 2016-04-07T03:53:31.298-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:31.47-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18210000 QueuedCount 50000 2016-04-07T03:53:31.906-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18220000 QueuedCount 48538 2016-04-07T03:53:32.323-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18230000 QueuedCount 38538 2016-04-07T03:53:32.745-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18240000 QueuedCount 28538 2016-04-07T03:53:33.153-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18250000 QueuedCount 18538 2016-04-07T03:53:33.564-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18260000 QueuedCount 8538 2016-04-07T03:53:34.419-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.959µs CommitTime 509.260388ms TotalFlushTime 12m34.505661542s TotalCommitTime 57.900150866s 2016-04-07T03:53:34.419-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 18268538, 18268538, 143 committed:true 2016-04-07T03:53:34.419-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 18268538, 18268538, 143 committed:true) SnapCreateDur 520.417842ms SnapOpenDur 65.507µs 2016-04-07T03:53:34.447-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:34.46-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18270000 QueuedCount 8707 2016-04-07T03:53:34.482-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:34.509-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:34.531-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:34.556-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:34.58-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:34.966-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18280000 QueuedCount 50000 2016-04-07T03:53:35.455-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18290000 QueuedCount 50000 2016-04-07T03:53:35.901-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18300000 QueuedCount 50000 2016-04-07T03:53:35.906-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:36.086-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:36.258-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:36.338-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18310000 QueuedCount 50000 2016-04-07T03:53:36.424-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:36.78-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18320000 QueuedCount 47513 2016-04-07T03:53:37.188-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18330000 QueuedCount 37513 2016-04-07T03:53:37.535-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18340000 QueuedCount 27513 2016-04-07T03:53:37.938-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18350000 QueuedCount 17513 2016-04-07T03:53:38.345-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18360000 QueuedCount 7513 2016-04-07T03:53:39.187-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.643µs CommitTime 510.106617ms TotalFlushTime 12m38.613789567s TotalCommitTime 58.410257483s 2016-04-07T03:53:39.187-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 18367513, 18367513, 144 committed:true 2016-04-07T03:53:39.187-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 18367513, 18367513, 144 committed:true) SnapCreateDur 521.178044ms SnapOpenDur 34.431µs 2016-04-07T03:53:39.209-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:39.247-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:39.272-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:39.297-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:39.316-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18370000 QueuedCount 44876 2016-04-07T03:53:39.338-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:39.358-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:39.716-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:39.806-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18380000 QueuedCount 50000 2016-04-07T03:53:39.914-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:40.23-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18390000 QueuedCount 50000 2016-04-07T03:53:40.418-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:41.589-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:41.608-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18400000 QueuedCount 48286 2016-04-07T03:53:42.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18410000 QueuedCount 43457 2016-04-07T03:53:42.479-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18420000 QueuedCount 33457 2016-04-07T03:53:42.887-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18430000 QueuedCount 23457 2016-04-07T03:53:43.292-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18440000 QueuedCount 13457 2016-04-07T03:53:43.705-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18450000 QueuedCount 3457 2016-04-07T03:53:44.36-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.978µs CommitTime 541.559814ms TotalFlushTime 12m43.116638763s TotalCommitTime 58.951817297s 2016-04-07T03:53:44.36-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 18453457, 18453457, 145 committed:true 2016-04-07T03:53:44.36-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 18453457, 18453457, 145 committed:true) SnapCreateDur 552.734659ms SnapOpenDur 61.429µs 2016-04-07T03:53:44.395-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:44.417-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:44.44-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:44.462-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:44.486-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:44.515-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:44.688-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18460000 QueuedCount 50000 2016-04-07T03:53:45.131-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18470000 QueuedCount 50000 2016-04-07T03:53:45.144-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:45.522-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18480000 QueuedCount 50000 2016-04-07T03:53:45.796-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:45.948-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:45.966-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18490000 QueuedCount 48301 2016-04-07T03:53:46.383-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:46.397-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18500000 QueuedCount 48681 2016-04-07T03:53:46.653-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:46.839-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18510000 QueuedCount 45881 2016-04-07T03:53:47.252-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18520000 QueuedCount 35881 2016-04-07T03:53:47.668-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18530000 QueuedCount 25881 2016-04-07T03:53:48.343-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18540000 QueuedCount 15881 2016-04-07T03:53:50.226-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18550000 QueuedCount 5881 2016-04-07T03:53:50.986-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.296µs CommitTime 572.232798ms TotalFlushTime 12m49.007128377s TotalCommitTime 59.524050095s 2016-04-07T03:53:50.986-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 18555881, 18555881, 146 committed:true 2016-04-07T03:53:50.986-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 18555881, 18555881, 146 committed:true) SnapCreateDur 582.56857ms SnapOpenDur 38.923µs 2016-04-07T03:53:51.009-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:51.032-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:51.053-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:51.084-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:51.105-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:51.136-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:51.159-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:51.18-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:51.199-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18560000 QueuedCount 46134 2016-04-07T03:53:51.497-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:51.611-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18570000 QueuedCount 50000 2016-04-07T03:53:52.739-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:52.994-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18580000 QueuedCount 44865 2016-04-07T03:53:53.389-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18590000 QueuedCount 34865 2016-04-07T03:53:53.782-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18600000 QueuedCount 24865 2016-04-07T03:53:54.192-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18610000 QueuedCount 14865 2016-04-07T03:53:54.604-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18620000 QueuedCount 4865 2016-04-07T03:53:55.262-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.473µs CommitTime 472.078643ms TotalFlushTime 12m52.717988148s TotalCommitTime 59.996128738s 2016-04-07T03:53:55.262-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 18624865, 18624865, 147 committed:true 2016-04-07T03:53:55.262-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 18624865, 18624865, 147 committed:true) SnapCreateDur 482.451486ms SnapOpenDur 52.679µs 2016-04-07T03:53:55.287-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:55.321-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:56.356-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18630000 QueuedCount 50000 2016-04-07T03:53:56.367-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:56.453-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:56.718-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:56.794-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18640000 QueuedCount 50000 2016-04-07T03:53:57.291-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18650000 QueuedCount 50000 2016-04-07T03:53:57.375-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:57.486-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:57.58-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:57.744-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18660000 QueuedCount 50000 2016-04-07T03:53:57.921-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:53:58.174-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18670000 QueuedCount 45077 2016-04-07T03:53:58.528-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18680000 QueuedCount 35077 2016-04-07T03:53:58.952-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18690000 QueuedCount 25077 2016-04-07T03:53:59.363-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18700000 QueuedCount 15077 2016-04-07T03:53:59.798-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18710000 QueuedCount 5077 2016-04-07T03:54:00.515-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.427µs CommitTime 462.711113ms TotalFlushTime 12m57.367601749s TotalCommitTime 1m0.458839851s 2016-04-07T03:54:00.515-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 18715077, 18715077, 148 committed:true 2016-04-07T03:54:00.515-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 18715077, 18715077, 148 committed:true) SnapCreateDur 473.332453ms SnapOpenDur 61.629µs 2016-04-07T03:54:00.536-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.557-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.577-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.598-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.619-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.639-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.66-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.681-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.702-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.724-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.747-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.774-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.795-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.818-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:00.84-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:01.483-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18720000 QueuedCount 50000 2016-04-07T03:54:01.665-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:01.952-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18730000 QueuedCount 50000 2016-04-07T03:54:01.963-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:02.415-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18740000 QueuedCount 50000 2016-04-07T03:54:02.703-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:02.724-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:02.921-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18750000 QueuedCount 50000 2016-04-07T03:54:03.892-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:04.161-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18760000 QueuedCount 47109 2016-04-07T03:54:04.556-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18770000 QueuedCount 37109 2016-04-07T03:54:04.969-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18780000 QueuedCount 27109 2016-04-07T03:54:06.163-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18790000 QueuedCount 17109 2016-04-07T03:54:06.575-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18800000 QueuedCount 7109 2016-04-07T03:54:07.365-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.262µs CommitTime 513.611352ms TotalFlushTime 13m3.54988228s TotalCommitTime 1m0.972451203s 2016-04-07T03:54:07.365-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 18807109, 18807109, 149 committed:true 2016-04-07T03:54:07.365-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 18807109, 18807109, 149 committed:true) SnapCreateDur 524.969487ms SnapOpenDur 37.359µs 2016-04-07T03:54:07.389-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.412-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.435-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.458-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.483-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.502-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18810000 QueuedCount 24559 2016-04-07T03:54:07.506-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.53-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.554-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.578-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.599-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.794-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.889-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:07.974-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18820000 QueuedCount 50000 2016-04-07T03:54:08.339-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:08.421-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18830000 QueuedCount 50000 2016-04-07T03:54:08.872-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18840000 QueuedCount 50000 2016-04-07T03:54:09.264-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18850000 QueuedCount 50000 2016-04-07T03:54:09.416-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:09.676-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18860000 QueuedCount 45929 2016-04-07T03:54:10.078-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18870000 QueuedCount 35929 2016-04-07T03:54:10.474-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18880000 QueuedCount 25929 2016-04-07T03:54:10.859-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18890000 QueuedCount 15929 2016-04-07T03:54:11.261-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18900000 QueuedCount 5929 2016-04-07T03:54:11.985-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.473µs CommitTime 476.148937ms TotalFlushTime 13m7.548637792s TotalCommitTime 1m1.44860014s 2016-04-07T03:54:11.985-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 18905929, 18905929, 150 committed:true 2016-04-07T03:54:11.985-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 18905929, 18905929, 150 committed:true) SnapCreateDur 486.921439ms SnapOpenDur 43.653µs 2016-04-07T03:54:12.006-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:12.03-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:12.054-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:12.077-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:12.101-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:12.125-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:12.216-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18910000 QueuedCount 31811 2016-04-07T03:54:12.221-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:12.549-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:12.724-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:12.739-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18920000 QueuedCount 48704 2016-04-07T03:54:12.815-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:12.945-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":1025634,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":5312612855,"bucket-1:myindex:avg_ts_items_count":94088,"bucket-1:myindex:build_progress":94,"bucket-1:myindex:data_size":6104880037,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":39453392896,"bucket-1:myindex:flush_queue_size":50011,"bucket-1:myindex:frag_percent":84,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":1985122545,"bucket-1:myindex:items_count":18807109,"bucket-1:myindex:num_commits":150,"bucket-1:myindex:num_compactions":3,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":18902283,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":1097717,"bucket-1:myindex:num_flush_queued":18974376,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":150,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":4620011295,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"150 61448600140 -8711797911773868304","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"18924366 121772761030 611033982096608902","bucket-1:myindex:timings/storage_handle_open":"150 3689235 95002728461","bucket-1:myindex:timings/storage_info":"601 5896047 156966242051","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"152 140836833 3663365539710529","bucket-1:myindex:timings/storage_meta_set":"150 386961778 107838819919283576","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"37848730 654407913160 1957700674064150530","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":57,"memory_quota":31457280000,"memory_used":6100606976,"needs_restart":false,"num_connections":0,"timings/stats_response":"3054 407238231 69927088727707"} 2016-04-07T03:54:13.016-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:13.199-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:13.215-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18930000 QueuedCount 48593 2016-04-07T03:54:13.298-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:13.318-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:13.339-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:13.36-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:13.39-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:13.677-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:13.682-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18940000 QueuedCount 49550 2016-04-07T03:54:13.698-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:14.119-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18950000 QueuedCount 50000 2016-04-07T03:54:14.482-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:14.56-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18960000 QueuedCount 50000 2016-04-07T03:54:15.027-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18970000 QueuedCount 50000 2016-04-07T03:54:15.444-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18980000 QueuedCount 43915 2016-04-07T03:54:15.838-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 18990000 QueuedCount 33915 2016-04-07T03:54:16.187-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19000000 QueuedCount 23915 2016-04-07T03:54:16.592-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19010000 QueuedCount 13915 2016-04-07T03:54:17.002-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19020000 QueuedCount 3915 2016-04-07T03:54:17.714-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.627µs CommitTime 531.738734ms TotalFlushTime 13m12.56196985s TotalCommitTime 1m1.980338874s 2016-04-07T03:54:17.714-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19023915, 19023915, 151 committed:true 2016-04-07T03:54:17.714-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19023915, 19023915, 151 committed:true) SnapCreateDur 542.836851ms SnapOpenDur 53.373µs 2016-04-07T03:54:17.74-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:17.844-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:17.962-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19030000 QueuedCount 50000 2016-04-07T03:54:18.429-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19040000 QueuedCount 50000 2016-04-07T03:54:18.913-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19050000 QueuedCount 50000 2016-04-07T03:54:19.276-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:19.365-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19060000 QueuedCount 50000 2016-04-07T03:54:19.453-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:19.551-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:19.815-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19070000 QueuedCount 49364 2016-04-07T03:54:20.241-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19080000 QueuedCount 39364 2016-04-07T03:54:20.647-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19090000 QueuedCount 29364 2016-04-07T03:54:21.041-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19100000 QueuedCount 19364 2016-04-07T03:54:21.431-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19110000 QueuedCount 9364 2016-04-07T03:54:23.152-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 10.309µs CommitTime 483.560812ms TotalFlushTime 13m17.36969045s TotalCommitTime 1m2.463899686s 2016-04-07T03:54:23.153-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19119364, 19119364, 152 committed:true 2016-04-07T03:54:23.153-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19119364, 19119364, 152 committed:true) SnapCreateDur 493.993295ms SnapOpenDur 50.284µs 2016-04-07T03:54:23.176-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:23.182-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19120000 QueuedCount 4060 2016-04-07T03:54:23.198-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:23.222-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:23.246-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:23.27-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:23.3-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:23.323-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:23.345-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:23.368-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:23.391-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:23.663-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19130000 QueuedCount 50000 2016-04-07T03:54:24.126-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19140000 QueuedCount 50000 2016-04-07T03:54:24.533-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19150000 QueuedCount 50000 2016-04-07T03:54:24.696-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:24.932-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19160000 QueuedCount 45336 2016-04-07T03:54:26.057-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19170000 QueuedCount 35336 2016-04-07T03:54:26.613-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19180000 QueuedCount 25336 2016-04-07T03:54:27.006-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19190000 QueuedCount 15336 2016-04-07T03:54:27.394-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19200000 QueuedCount 5336 2016-04-07T03:54:28.06-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.341µs CommitTime 487.042796ms TotalFlushTime 13m21.66440236s TotalCommitTime 1m2.950942482s 2016-04-07T03:54:28.06-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19205336, 19205336, 153 committed:true 2016-04-07T03:54:28.06-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19205336, 19205336, 153 committed:true) SnapCreateDur 496.944076ms SnapOpenDur 49.006µs 2016-04-07T03:54:28.104-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:28.125-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:28.292-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19210000 QueuedCount 50000 2016-04-07T03:54:28.385-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:28.768-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19220000 QueuedCount 50000 2016-04-07T03:54:29.218-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19230000 QueuedCount 50000 2016-04-07T03:54:29.6-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19240000 QueuedCount 50000 2016-04-07T03:54:30.038-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19250000 QueuedCount 50000 2016-04-07T03:54:30.278-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:30.462-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19260000 QueuedCount 47699 2016-04-07T03:54:30.873-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19270000 QueuedCount 37699 2016-04-07T03:54:31.279-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19280000 QueuedCount 27699 2016-04-07T03:54:32.036-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19290000 QueuedCount 17699 2016-04-07T03:54:33.057-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19300000 QueuedCount 7699 2016-04-07T03:54:33.888-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.032µs CommitTime 563.278874ms TotalFlushTime 13m26.759171228s TotalCommitTime 1m3.514221356s 2016-04-07T03:54:33.888-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19307699, 19307699, 154 committed:true 2016-04-07T03:54:33.888-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19307699, 19307699, 154 committed:true) SnapCreateDur 573.939608ms SnapOpenDur 37.074µs 2016-04-07T03:54:33.914-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:33.938-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:33.963-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:34.014-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19310000 QueuedCount 50000 2016-04-07T03:54:34.128-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:35.51-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19320000 QueuedCount 50000 2016-04-07T03:54:35.909-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19330000 QueuedCount 50000 2016-04-07T03:54:36.365-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19340000 QueuedCount 50000 2016-04-07T03:54:36.816-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19350000 QueuedCount 50000 2016-04-07T03:54:38.308-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19360000 QueuedCount 50000 2016-04-07T03:54:38.571-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:38.747-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19370000 QueuedCount 50000 2016-04-07T03:54:39.182-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19380000 QueuedCount 50000 2016-04-07T03:54:39.623-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19390000 QueuedCount 50000 2016-04-07T03:54:41.069-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19400000 QueuedCount 50000 2016-04-07T03:54:41.434-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19410000 QueuedCount 47819 2016-04-07T03:54:41.81-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19420000 QueuedCount 37819 2016-04-07T03:54:42.19-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19430000 QueuedCount 27819 2016-04-07T03:54:42.535-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19440000 QueuedCount 17819 2016-04-07T03:54:42.915-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19450000 QueuedCount 7819 2016-04-07T03:54:43.748-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.608µs CommitTime 556.956548ms TotalFlushTime 13m35.804005779s TotalCommitTime 1m4.071177904s 2016-04-07T03:54:43.748-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19457819, 19457819, 155 committed:true 2016-04-07T03:54:43.748-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19457819, 19457819, 155 committed:true) SnapCreateDur 566.837362ms SnapOpenDur 35.189µs 2016-04-07T03:54:44.796-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19460000 QueuedCount 39185 2016-04-07T03:54:45.145-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19470000 QueuedCount 29185 2016-04-07T03:54:45.539-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19480000 QueuedCount 19185 2016-04-07T03:54:45.937-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19490000 QueuedCount 9185 2016-04-07T03:54:46.64-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.748µs CommitTime 317.652383ms TotalFlushTime 13m38.320243613s TotalCommitTime 1m4.388830287s 2016-04-07T03:54:46.64-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19499185, 19499185, 156 committed:true 2016-04-07T03:54:46.64-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19499185, 19499185, 156 committed:true) SnapCreateDur 327.349275ms SnapOpenDur 39.225µs 2016-04-07T03:54:46.724-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19500000 QueuedCount 32043 2016-04-07T03:54:46.724-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:47.245-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19510000 QueuedCount 50000 2016-04-07T03:54:47.447-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:47.741-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19520000 QueuedCount 50000 2016-04-07T03:54:48.211-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19530000 QueuedCount 50000 2016-04-07T03:54:48.604-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19540000 QueuedCount 50000 2016-04-07T03:54:48.778-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:49.071-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19550000 QueuedCount 50000 2016-04-07T03:54:49.538-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19560000 QueuedCount 50000 2016-04-07T03:54:49.997-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19570000 QueuedCount 50000 2016-04-07T03:54:50.434-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19580000 QueuedCount 42034 2016-04-07T03:54:50.847-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19590000 QueuedCount 32034 2016-04-07T03:54:51.278-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19600000 QueuedCount 22034 2016-04-07T03:54:51.69-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19610000 QueuedCount 12034 2016-04-07T03:54:52.044-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19620000 QueuedCount 2034 2016-04-07T03:54:52.742-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.801µs CommitTime 592.481789ms TotalFlushTime 13m43.604774513s TotalCommitTime 1m4.981312076s 2016-04-07T03:54:52.742-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19622034, 19622034, 157 committed:true 2016-04-07T03:54:52.742-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19622034, 19622034, 157 committed:true) SnapCreateDur 603.503344ms SnapOpenDur 40.582µs 2016-04-07T03:54:52.775-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:52.812-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:53.098-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19630000 QueuedCount 50000 2016-04-07T03:54:53.56-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19640000 QueuedCount 50000 2016-04-07T03:54:55.04-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19650000 QueuedCount 50000 2016-04-07T03:54:55.465-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19660000 QueuedCount 50000 2016-04-07T03:54:55.868-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19670000 QueuedCount 50000 2016-04-07T03:54:56.231-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:54:56.311-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19680000 QueuedCount 50000 2016-04-07T03:54:56.737-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19690000 QueuedCount 50000 2016-04-07T03:54:57.155-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19700000 QueuedCount 50000 2016-04-07T03:54:57.519-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19710000 QueuedCount 50000 2016-04-07T03:54:58.98-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19720000 QueuedCount 50000 2016-04-07T03:54:59.364-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19730000 QueuedCount 43305 2016-04-07T03:54:59.737-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19740000 QueuedCount 33305 2016-04-07T03:55:00.094-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19750000 QueuedCount 23305 2016-04-07T03:55:00.466-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19760000 QueuedCount 13305 2016-04-07T03:55:00.834-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19770000 QueuedCount 3305 2016-04-07T03:55:01.513-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.552µs CommitTime 575.860151ms TotalFlushTime 13m51.534920245s TotalCommitTime 1m5.557172227s 2016-04-07T03:55:01.513-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19773305, 19773305, 158 committed:true 2016-04-07T03:55:01.513-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19773305, 19773305, 158 committed:true) SnapCreateDur 586.430064ms SnapOpenDur 45.599µs 2016-04-07T03:55:01.804-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19780000 QueuedCount 38355 2016-04-07T03:55:02.214-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19790000 QueuedCount 28355 2016-04-07T03:55:02.599-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19800000 QueuedCount 18355 2016-04-07T03:55:03.502-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19810000 QueuedCount 8355 2016-04-07T03:55:04.624-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.729µs CommitTime 383.651729ms TotalFlushTime 13m54.201723037s TotalCommitTime 1m5.940823956s 2016-04-07T03:55:04.624-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19818355, 19818355, 159 committed:true 2016-04-07T03:55:04.624-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19818355, 19818355, 159 committed:true) SnapCreateDur 394.717756ms SnapOpenDur 37.831µs 2016-04-07T03:55:04.67-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:55:05.267-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19820000 QueuedCount 50000 2016-04-07T03:55:05.742-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19830000 QueuedCount 50000 2016-04-07T03:55:06.201-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19840000 QueuedCount 50000 2016-04-07T03:55:06.633-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19850000 QueuedCount 50000 2016-04-07T03:55:07.041-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19860000 QueuedCount 50000 2016-04-07T03:55:07.131-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:55:07.466-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19870000 QueuedCount 50000 2016-04-07T03:55:07.935-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19880000 QueuedCount 50000 2016-04-07T03:55:08.308-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19890000 QueuedCount 50000 2016-04-07T03:55:08.73-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19900000 QueuedCount 50000 2016-04-07T03:55:09.148-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19910000 QueuedCount 46266 2016-04-07T03:55:10.452-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19920000 QueuedCount 36266 2016-04-07T03:55:10.822-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19930000 QueuedCount 26266 2016-04-07T03:55:11.196-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19940000 QueuedCount 16266 2016-04-07T03:55:11.549-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19950000 QueuedCount 6266 2016-04-07T03:55:12.344-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.669µs CommitTime 557.199954ms TotalFlushTime 14m1.130069972s TotalCommitTime 1m6.49802391s 2016-04-07T03:55:12.344-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19956266, 19956266, 160 committed:true 2016-04-07T03:55:12.344-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19956266, 19956266, 160 committed:true) SnapCreateDur 567.02343ms SnapOpenDur 46.547µs 2016-04-07T03:55:12.485-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19960000 QueuedCount 38452 2016-04-07T03:55:12.915-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19970000 QueuedCount 28452 2016-04-07T03:55:12.945-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":1548,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":6446231959,"bucket-1:myindex:avg_ts_items_count":111972,"bucket-1:myindex:build_progress":99,"bucket-1:myindex:data_size":6442309298,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":42987171840,"bucket-1:myindex:flush_queue_size":27847,"bucket-1:myindex:frag_percent":85,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":2095407930,"bucket-1:myindex:items_count":19818355,"bucket-1:myindex:num_commits":160,"bucket-1:myindex:num_compactions":3,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":19912431,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":87569,"bucket-1:myindex:num_flush_queued":19998452,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":160,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":7720466339,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"160 66498023910 -6089313342111310840","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"19970606 128553286023 612130431550554715","bucket-1:myindex:timings/storage_handle_open":"160 3935719 101390316707","bucket-1:myindex:timings/storage_info":"641 6267837 167127315509","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"162 144105843 3664458127341561","bucket-1:myindex:timings/storage_meta_set":"160 390660107 107840243642648949","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"39941210 700149187070 3789095040491895336","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":4,"memory_quota":31457280000,"memory_used":7841325056,"needs_restart":false,"num_connections":0,"timings/stats_response":"3197 432436427 74522258079981"} 2016-04-07T03:55:13.298-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19980000 QueuedCount 18452 2016-04-07T03:55:13.678-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 19990000 QueuedCount 8452 2016-04-07T03:55:14.502-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.226µs CommitTime 484.455382ms TotalFlushTime 14m2.750086642s TotalCommitTime 1m6.982479292s 2016-04-07T03:55:14.502-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 19998452, 19998452, 161 committed:true 2016-04-07T03:55:14.502-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 19998452, 19998452, 161 committed:true) SnapCreateDur 495.529313ms SnapOpenDur 38.531µs 2016-04-07T03:55:14.526-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:55:14.546-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:55:14.567-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T03:55:14.589-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20000000 QueuedCount 0 2016-04-07T03:55:14.688-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.144µs CommitTime 79.744794ms TotalFlushTime 14m2.766501005s TotalCommitTime 1m7.062224086s 2016-04-07T03:55:14.689-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20000000, 20000000, 162 committed:true 2016-04-07T03:55:14.689-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20000000, 20000000, 162 committed:true) SnapCreateDur 90.376941ms SnapOpenDur 38.194µs 2016-04-07T03:55:14.689-07:00 [Info] Timekeeper::checkInitialBuildDone remove index 9366861204008862690 from stream MAINT_STREAM bucket bucket-1 2016-04-07T03:55:14.689-07:00 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 9366861204008862690 Stream: MAINT_STREAM Bucket: bucket-1 BuildTS: [0=19535,1=19525,2=19535,3=19530,4=19535,5=19530,6=19535,7=19525,8=19535,9=19525,10=19535,11=19530,12=19535,13=19530,14=19535,15=19525,16=19535,17=19525,18=19536,19=19530,20=19535,21=19530,22=19535,23=19525,24=19535,25=19525,26=19535,27=19530,28=19535,29=19530,30=19535,31=19525,32=19535,33=19525,34=19535,35=19530,36=19535,37=19530,38=19535,39=19525,40=19535,41=19525,42=19535,43=19530,44=19535,45=19530,46=19535,47=19525,48=19535,49=19525,50=19535,51=19530,52=19535,53=19530,54=19535,55=19525,56=19535,57=19525,58=19535,59=19530,60=19535,61=19530,62=19535,63=19525,64=19530,65=19535,66=19525,67=19535,68=19525,69=19535,70=19530,71=19535,72=19530,73=19535,74=19525,75=19535,76=19525,77=19535,78=19530,79=19535,80=19530,81=19535,82=19525,83=19535,84=19525,85=19535,86=19530,87=19535,88=19530,89=19535,90=19525,91=19535,92=19525,93=19535,94=19530,95=19535,96=19530,97=19535,98=19525,99=19535,100=19525,101=19535,102=19530,103=19535,104=19530,105=19535,106=19525,107=19535,108=19525,109=19535,110=19530,111=19535,112=19530,113=19535,114=19525,115=19535,116=19525,117=19535,118=19530,119=19535,120=19530,121=19535,122=19525,123=19535,124=19525,125=19535,126=19530,127=19535,128=19535,129=19530,130=19535,131=19525,132=19535,133=19525,134=19535,135=19530,136=19535,137=19530,138=19535,139=19525,140=19535,141=19525,142=19535,143=19530,144=19535,145=19530,146=19535,147=19525,148=19535,149=19525,150=19535,151=19530,152=19535,153=19530,154=19535,155=19525,156=19535,157=19525,158=19535,159=19530,160=19535,161=19530,162=19535,163=19525,164=19535,165=19525,166=19535,167=19530,168=19535,169=19530,170=19535,171=19525,172=19535,173=19525,174=19535,175=19530,176=19535,177=19530,178=19535,179=19525,180=19535,181=19525,182=19535,183=19530,184=19535,185=19530,186=19535,187=19525,188=19535,189=19525,190=19535,191=19530,192=19525,193=19535,194=19530,195=19535,196=19530,197=19535,198=19525,199=19535,200=19525,201=19535,202=19530,203=19535,204=19530,205=19535,206=19525,207=19535,208=19525,209=19535,210=19530,211=19535,212=19530,213=19535,214=19525,215=19535,216=19525,217=19535,218=19530,219=19535,220=19530,221=19535,222=19525,223=19535,224=19525,225=19535,226=19530,227=19535,228=19530,229=19535,230=19525,231=19535,232=19525,233=19535,234=19530,235=19535,236=19530,237=19535,238=19525,239=19535,240=19525,241=19535,242=19530,243=19535,244=19530,245=19535,246=19525,247=19535,248=19525,249=19535,250=19530,251=19535,252=19530,253=19535,254=19525,255=19535,256=19530,257=19535,258=19525,259=19535,260=19525,261=19535,262=19530,263=19535,264=19530,265=19535,266=19525,267=19535,268=19525,269=19535,270=19530,271=19535,272=19530,273=19535,274=19525,275=19535,276=19525,277=19535,278=19530,279=19535,280=19530,281=19535,282=19525,283=19535,284=19525,285=19535,286=19530,287=19535,288=19530,289=19535,290=19525,291=19535,292=19525,293=19535,294=19530,295=19535,296=19530,297=19535,298=19525,299=19535,300=19525,301=19535,302=19530,303=19535,304=19530,305=19535,306=19525,307=19535,308=19525,309=19535,310=19530,311=19535,312=19530,313=19535,314=19525,315=19535,316=19525,317=19535,318=19530,319=19535,320=19535,321=19525,322=19535,323=19530,324=19535,325=19530,326=19535,327=19525,328=19535,329=19525,330=19535,331=19530,332=19535,333=19530,334=19535,335=19525,336=19535,337=19525,338=19535,339=19530,340=19535,341=19530,342=19535,343=19525,344=19535,345=19525,346=19535,347=19530,348=19535,349=19530,350=19535,351=19525,352=19535,353=19525,354=19535,355=19530,356=19535,357=19530,358=19535,359=19525,360=19535,361=19525,362=19535,363=19530,364=19535,365=19530,366=19535,367=19525,368=19535,369=19525,370=19535,371=19530,372=19535,373=19530,374=19535,375=19525,376=19535,377=19525,378=19535,379=19530,380=19535,381=19530,382=19535,383=19525,384=19525,385=19535,386=19530,387=19535,388=19530,389=19535,390=19525,391=19535,392=19525,393=19535,394=19530,395=19535,396=19530,397=19535,398=19525,399=19535,400=19525,401=19535,402=19530,403=19535,404=19530,405=19535,406=19525,407=19535,408=19525,409=19535,410=19530,411=19535,412=19530,413=19535,414=19525,415=19535,416=19525,417=19535,418=19530,419=19535,420=19530,421=19535,422=19525,423=19535,424=19525,425=19535,426=19530,427=19535,428=19530,429=19535,430=19525,431=19535,432=19525,433=19535,434=19530,435=19535,436=19530,437=19535,438=19525,439=19535,440=19525,441=19535,442=19530,443=19535,444=19530,445=19535,446=19525,447=19535,448=19535,449=19530,450=19535,451=19525,452=19535,453=19525,454=19535,455=19530,456=19535,457=19530,458=19535,459=19525,460=19535,461=19525,462=19535,463=19530,464=19535,465=19530,466=19535,467=19525,468=19535,469=19525,470=19535,471=19530,472=19535,473=19530,474=19535,475=19525,476=19535,477=19525,478=19535,479=19530,480=19535,481=19530,482=19535,483=19525,484=19535,485=19525,486=19535,487=19530,488=19535,489=19530,490=19535,491=19525,492=19535,493=19525,494=19535,495=19530,496=19535,497=19530,498=19535,499=19525,500=19535,501=19525,502=19535,503=19530,504=19535,505=19530,506=19535,507=19525,508=19535,509=19525,510=19535,511=19530,512=19535,513=19530,514=19535,515=19525,516=19535,517=19525,518=19535,519=19530,520=19535,521=19530,522=19535,523=19525,524=19535,525=19525,526=19535,527=19530,528=19535,529=19530,530=19535,531=19525,532=19535,533=19525,534=19535,535=19530,536=19535,537=19530,538=19535,539=19525,540=19535,541=19525,542=19535,543=19530,544=19535,545=19530,546=19535,547=19525,548=19535,549=19525,550=19535,551=19530,552=19535,553=19530,554=19535,555=19525,556=19535,557=19525,558=19535,559=19530,560=19535,561=19530,562=19535,563=19525,564=19535,565=19525,566=19535,567=19529,568=19535,569=19530,570=19535,571=19525,572=19535,573=19525,574=19535,575=19530,576=19525,577=19535,578=19530,579=19535,580=19530,581=19535,582=19525,583=19535,584=19525,585=19535,586=19530,587=19535,588=19530,589=19535,590=19525,591=19535,592=19525,593=19535,594=19530,595=19535,596=19530,597=19535,598=19525,599=19535,600=19525,601=19535,602=19530,603=19535,604=19530,605=19535,606=19525,607=19535,608=19525,609=19535,610=19530,611=19535,612=19530,613=19535,614=19525,615=19535,616=19525,617=19535,618=19530,619=19535,620=19530,621=19535,622=19525,623=19535,624=19525,625=19535,626=19530,627=19535,628=19530,629=19535,630=19525,631=19535,632=19525,633=19535,634=19530,635=19535,636=19530,637=19535,638=19525,639=19535,640=19535,641=19525,642=19535,643=19530,644=19535,645=19530,646=19535,647=19525,648=19535,649=19525,650=19535,651=19530,652=19535,653=19530,654=19535,655=19525,656=19535,657=19525,658=19535,659=19530,660=19535,661=19530,662=19535,663=19525,664=19535,665=19525,666=19535,667=19530,668=19535,669=19530,670=19535,671=19525,672=19535,673=19525,674=19535,675=19530,676=19535,677=19530,678=19535,679=19525,680=19535,681=19525,682=19535,683=19530,684=19535,685=19530,686=19535,687=19525,688=19535,689=19525,690=19535,691=19530,692=19535,693=19530,694=19535,695=19525,696=19535,697=19525,698=19535,699=19530,700=19535,701=19530,702=19535,703=19525,704=19530,705=19535,706=19525,707=19535,708=19525,709=19535,710=19530,711=19535,712=19530,713=19535,714=19525,715=19535,716=19525,717=19535,718=19530,719=19535,720=19530,721=19535,722=19525,723=19535,724=19525,725=19535,726=19530,727=19535,728=19530,729=19535,730=19525,731=19535,732=19525,733=19535,734=19530,735=19535,736=19530,737=19535,738=19525,739=19535,740=19525,741=19535,742=19530,743=19535,744=19530,745=19535,746=19525,747=19535,748=19525,749=19535,750=19530,751=19535,752=19530,753=19535,754=19525,755=19535,756=19525,757=19535,758=19530,759=19535,760=19530,761=19535,762=19525,763=19535,764=19525,765=19535,766=19530,767=19535,768=19525,769=19535,770=19530,771=19535,772=19530,773=19535,774=19525,775=19535,776=19525,777=19535,778=19530,779=19535,780=19530,781=19535,782=19525,783=19535,784=19525,785=19535,786=19530,787=19535,788=19530,789=19535,790=19525,791=19535,792=19525,793=19535,794=19530,795=19535,796=19530,797=19535,798=19525,799=19535,800=19525,801=19535,802=19530,803=19535,804=19530,805=19535,806=19525,807=19535,808=19525,809=19535,810=19530,811=19535,812=19530,813=195 35,814=19525,815=19535,816=19525,817=19535,818=19530,819=19535,820=19530,821=19535,822=19525,823=19535,824=19525,825=19535,826=19530,827=19535,828=19530,829=19535,830=19525,831=19535,832=19535,833=19530,834=19535,835=19525,836=19535,837=19525,838=19535,839=19530,840=19535,841=19530,842=19535,843=19525,844=19535,845=19525,846=19535,847=19530,848=19535,849=19530,850=19535,851=19525,852=19535,853=19525,854=19535,855=19530,856=19535,857=19530,858=19535,859=19525,860=19535,861=19525,862=19535,863=19530,864=19535,865=19530,866=19535,867=19525,868=19535,869=19525,870=19535,871=19530,872=19535,873=19530,874=19535,875=19525,876=19535,877=19525,878=19535,879=19530,880=19535,881=19530,882=19535,883=19525,884=19535,885=19525,886=19535,887=19530,888=19535,889=19530,890=19535,891=19525,892=19535,893=19525,894=19535,895=19530,896=19530,897=19535,898=19525,899=19535,900=19525,901=19535,902=19530,903=19535,904=19530,905=19535,906=19525,907=19535,908=19525,909=19535,910=19530,911=19535,912=19530,913=19535,914=19525,915=19535,916=19525,917=19535,918=19530,919=19535,920=19530,921=19535,922=19525,923=19535,924=19525,925=19535,926=19530,927=19535,928=19530,929=19535,930=19525,931=19535,932=19525,933=19535,934=19530,935=19535,936=19530,937=19535,938=19525,939=19535,940=19525,941=19535,942=19530,943=19535,944=19530,945=19535,946=19525,947=19535,948=19525,949=19535,950=19530,951=19535,952=19530,953=19535,954=19525,955=19535,956=19525,957=19535,958=19530,959=19535,960=19535,961=19525,962=19535,963=19530,964=19535,965=19530,966=19535,967=19525,968=19535,969=19525,970=19535,971=19530,972=19535,973=19530,974=19535,975=19525,976=19535,977=19525,978=19535,979=19530,980=19535,981=19530,982=19535,983=19525,984=19535,985=19525,986=19535,987=19530,988=19535,989=19530,990=19535,991=19525,992=19535,993=19525,994=19535,995=19530,996=19535,997=19530,998=19535,999=19525,1000=19535,1001=19525,1002=19535,1003=19530,1004=19535,1005=19530,1006=19535,1007=19525,1008=19535,1009=19525,1010=19535,1011=19530,1012=19535,1013=19530,1014=19535,1015=19525,1016=19535,1017=19525,1018=19535,1019=19530,1020=19535,1021=19530,1022=19535,1023=19525] 2016-04-07T03:55:14.691-07:00 [Info] Indexer::handleInitialBuildDone Bucket: bucket-1 Stream: MAINT_STREAM 2016-04-07T03:55:14.696-07:00 [Info] MutationMgr::handleUpdateIndexInstMap Message: MsgUpdateInstMap InstanceId: 9366861204008862690 Name: myindex Bucket: bucket-1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM 2016-04-07T03:55:14.696-07:00 [Info] ClustMgr:handleUpdateTopologyForIndex &{47 [{9366861204008862690 {9366861204008862690 myindex forestdb bucket-1 8bc620dc3c466302736c51871ada764f false [`alt_email`] N1QL SINGLE partn true [a0:85:3b:34:1d:f9:92:ae]} 4 1 0xc20bfb35e0 []}] {true true false false} 0} 2016-04-07T03:56:12.945-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2244202486,"bucket-1:myindex:avg_ts_items_count":39313,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6457262379,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":43244154880,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":85,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":2100000000,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":162,"bucket-1:myindex:num_compactions":3,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":20000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":20000000,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":162,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":186301928,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"162 67062224086 -5848257092792442480","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"20000000 128720971401 612131435420632491","bucket-1:myindex:timings/storage_handle_open":"162 3980995 102420337357","bucket-1:myindex:timings/storage_info":"649 6329153 168190358817","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"164 144760939 3664672849856027","bucket-1:myindex:timings/storage_meta_set":"162 391455349 107840562507835913","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"40000000 701054427830 3834762899637366244","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":9009459200,"needs_restart":false,"num_connections":0,"timings/stats_response":"3272 444244491 76459282995603"} 2016-04-07T03:57:12.945-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2244202486,"bucket-1:myindex:avg_ts_items_count":39313,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6457262379,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":43244154880,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":85,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":2100000000,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":162,"bucket-1:myindex:num_compactions":3,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":20000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":20000000,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":162,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":186301928,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"162 67062224086 -5848257092792442480","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"20000000 128720971401 612131435420632491","bucket-1:myindex:timings/storage_handle_open":"162 3980995 102420337357","bucket-1:myindex:timings/storage_info":"649 6329153 168190358817","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"164 144760939 3664672849856027","bucket-1:myindex:timings/storage_meta_set":"162 391455349 107840562507835913","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"40000000 701054427830 3834762899637366244","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":10207547392,"needs_restart":false,"num_connections":0,"timings/stats_response":"3344 455575612 78300699774900"} 2016-04-07T03:58:12.945-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2244202486,"bucket-1:myindex:avg_ts_items_count":39313,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6457262379,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":43244154880,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":85,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":2100000000,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":162,"bucket-1:myindex:num_compactions":3,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":20000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":20000000,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":162,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":186301928,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"162 67062224086 -5848257092792442480","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"20000000 128720971401 612131435420632491","bucket-1:myindex:timings/storage_handle_open":"162 3980995 102420337357","bucket-1:myindex:timings/storage_info":"649 6329153 168190358817","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"164 144760939 3664672849856027","bucket-1:myindex:timings/storage_meta_set":"162 391455349 107840562507835913","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"40000000 701054427830 3834762899637366244","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":11375005696,"needs_restart":false,"num_connections":0,"timings/stats_response":"3416 466470151 80012762543109"} 2016-04-07T03:59:12.946-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2244202486,"bucket-1:myindex:avg_ts_items_count":39313,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6457262379,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":43244154880,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":85,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":2100000000,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":162,"bucket-1:myindex:num_compactions":3,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":20000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":20000000,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":162,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":186301928,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"162 67062224086 -5848257092792442480","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"20000000 128720971401 612131435420632491","bucket-1:myindex:timings/storage_handle_open":"162 3980995 102420337357","bucket-1:myindex:timings/storage_info":"649 6329153 168190358817","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"164 144760939 3664672849856027","bucket-1:myindex:timings/storage_meta_set":"162 391455349 107840562507835913","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"40000000 701054427830 3834762899637366244","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":14480867328,"needs_restart":false,"num_connections":0,"timings/stats_response":"3487 476838542 81557193665866"} 2016-04-07T04:00:12.946-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2244202486,"bucket-1:myindex:avg_ts_items_count":39313,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6457262379,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":43244154880,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":85,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":2100000000,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":162,"bucket-1:myindex:num_compactions":3,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":20000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":20000000,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":162,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":186301928,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"162 67062224086 -5848257092792442480","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"20000000 128720971401 612131435420632491","bucket-1:myindex:timings/storage_handle_open":"162 3980995 102420337357","bucket-1:myindex:timings/storage_info":"649 6329153 168190358817","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"164 144760939 3664672849856027","bucket-1:myindex:timings/storage_meta_set":"162 391455349 107840562507835913","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"40000000 701054427830 3834762899637366244","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"3559 488283503 83485971972601"} 2016-04-07T04:00:59.527-07:00 [Info] CompactionDaemon: Finished compacting index instance:9366861204008862690 2016-04-07T04:01:12.946-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2244202486,"bucket-1:myindex:avg_ts_items_count":39313,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6576591147,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":17268740096,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":61,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":2100000000,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":162,"bucket-1:myindex:num_compactions":3,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":20000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":20000000,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":162,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":186301928,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"162 67062224086 -5848257092792442480","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"20000000 128720971401 612131435420632491","bucket-1:myindex:timings/storage_handle_open":"162 3980995 102420337357","bucket-1:myindex:timings/storage_info":"649 6329153 168190358817","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"164 144760939 3664672849856027","bucket-1:myindex:timings/storage_meta_set":"162 391455349 107840562507835913","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"40000000 701054427830 3834762899637366244","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"3631 498780847 85061359527385"} 2016-04-07T04:01:29.528-07:00 [Info] CompactionDaemon: Checking fragmentation, IndexInstId: 9366861204008862690 Data:6576591147, Disk:17268740096, ExtraSnapshotData:0, Fragmentation:61.91620749145821% 2016-04-07T04:01:29.528-07:00 [Info] CompactionDaemon: Compacting index instance:9366861204008862690 2016-04-07T04:01:29.537-07:00 [Info] ForestDBSlice::Compact Compacting upto SeqNum 19773305. Slice Id 0, IndexInstId 9366861204008862690, IndexDefnId 9366861204008862690 2016-04-07T04:02:12.946-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2244202486,"bucket-1:myindex:avg_ts_items_count":39313,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6576591147,"bucket-1:myindex:delete_bytes":0,"bucket-1:myindex:disk_size":17268748288,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":61,"bucket-1:myindex:get_bytes":0,"bucket-1:myindex:insert_bytes":2100000000,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":162,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":20000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":20000000,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":162,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":186301928,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"162 67062224086 -5848257092792442480","bucket-1:myindex:timings/storage_del":"0 0 0","bucket-1:myindex:timings/storage_get":"20000000 128720971401 612131435420632491","bucket-1:myindex:timings/storage_handle_open":"162 3980995 102420337357","bucket-1:myindex:timings/storage_info":"649 6329153 168190358817","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"165 145229979 3664892848377627","bucket-1:myindex:timings/storage_meta_set":"162 391455349 107840562507835913","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"40000000 701054427830 3834762899637366244","bucket-1:myindex:timings/storage_snapshot_close":"0 0 0","bucket-1:myindex:timings/storage_snapshot_create":"0 0 0","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":20000000,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"3703 511188712 87263977062022"} 2016-04-07T04:02:17.073-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.535µs CommitTime 35.357198ms TotalFlushTime 14m2.78025595s TotalCommitTime 1m7.097581284s 2016-04-07T04:02:17.073-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20002402, 20001201, 163 committed:true 2016-04-07T04:02:17.073-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20002402, 20001201, 163 committed:true) SnapCreateDur 46.529881ms SnapOpenDur 66.596µs 2016-04-07T04:02:17.261-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20010000 2016-04-07T04:02:17.359-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20013890, 20006945, 0 committed:false 2016-04-07T04:02:17.664-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20020000 2016-04-07T04:02:17.963-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20030000 2016-04-07T04:02:18.325-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20040000 2016-04-07T04:02:18.652-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20050000 2016-04-07T04:02:18.973-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20060000 2016-04-07T04:02:19.286-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20070000 2016-04-07T04:02:19.562-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20080000 2016-04-07T04:02:19.874-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20090000 2016-04-07T04:02:20.138-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20100000 2016-04-07T04:02:20.46-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20110000 2016-04-07T04:02:20.721-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20120000 2016-04-07T04:02:21.031-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20130000 2016-04-07T04:02:21.324-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20140000 2016-04-07T04:02:21.572-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20150000 2016-04-07T04:02:21.876-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20160000 2016-04-07T04:02:22.123-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20170000 2016-04-07T04:02:22.375-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20180000 2016-04-07T04:02:22.69-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20190000 2016-04-07T04:02:22.893-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20013890, 20006945, 0 committed:false) SnapCreateDur 14.707µs SnapOpenDur 5.533836563s 2016-04-07T04:02:22.916-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20200000 2016-04-07T04:02:23.021-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20010000 QueuedCount 1712 2016-04-07T04:02:23.199-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20210000 2016-04-07T04:02:23.218-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.602µs CommitTime 89.913405ms TotalFlushTime 14m3.118699833s TotalCommitTime 1m7.187494689s 2016-04-07T04:02:23.218-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20023058, 20011529, 164 committed:true 2016-04-07T04:02:23.218-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20023058, 20011529, 164 committed:true) SnapCreateDur 102.374042ms SnapOpenDur 61.943µs 2016-04-07T04:02:23.241-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:23.265-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:23.5-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20220000 2016-04-07T04:02:23.571-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20020000 QueuedCount 9542 2016-04-07T04:02:23.722-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20230000 2016-04-07T04:02:23.928-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20058184, 20029092, 0 committed:false 2016-04-07T04:02:23.984-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20058184, 20029092, 0 committed:false) SnapCreateDur 13.97µs SnapOpenDur 56.15776ms 2016-04-07T04:02:24.008-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:24.008-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20240000 2016-04-07T04:02:24.013-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20030000 QueuedCount 3648 2016-04-07T04:02:24.031-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:24.321-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20250000 2016-04-07T04:02:24.536-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20040000 QueuedCount 7240 2016-04-07T04:02:24.551-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20260000 2016-04-07T04:02:24.846-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20270000 2016-04-07T04:02:24.885-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20093002, 20046501, 0 committed:false 2016-04-07T04:02:24.983-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20093002, 20046501, 0 committed:false) SnapCreateDur 12.1µs SnapOpenDur 97.870856ms 2016-04-07T04:02:25.007-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:25.03-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:25.12-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20050000 QueuedCount 15973 2016-04-07T04:02:25.141-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20280000 2016-04-07T04:02:25.385-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20290000 2016-04-07T04:02:25.6-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20060000 QueuedCount 5973 2016-04-07T04:02:25.625-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20300000 2016-04-07T04:02:25.894-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20129828, 20064914, 0 committed:false 2016-04-07T04:02:25.898-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20310000 2016-04-07T04:02:26.028-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20129828, 20064914, 0 committed:false) SnapCreateDur 13.592µs SnapOpenDur 134.344919ms 2016-04-07T04:02:26.052-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:26.109-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:26.179-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20320000 2016-04-07T04:02:26.268-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20070000 QueuedCount 15206 2016-04-07T04:02:26.399-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20330000 2016-04-07T04:02:26.657-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20340000 2016-04-07T04:02:26.767-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20080000 QueuedCount 5206 2016-04-07T04:02:26.966-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20350000 2016-04-07T04:02:27.053-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20167638, 20083819, 0 committed:false 2016-04-07T04:02:27.181-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20360000 2016-04-07T04:02:27.223-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20167638, 20083819, 0 committed:false) SnapCreateDur 9.968µs SnapOpenDur 170.019647ms 2016-04-07T04:02:27.247-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:27.271-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:27.442-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20370000 2016-04-07T04:02:27.52-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20090000 QueuedCount 16588 2016-04-07T04:02:27.751-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20380000 2016-04-07T04:02:27.968-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20390000 2016-04-07T04:02:28.004-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20100000 QueuedCount 6588 2016-04-07T04:02:28.179-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20400000 2016-04-07T04:02:28.365-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20209724, 20104862, 0 committed:false 2016-04-07T04:02:28.461-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20410000 2016-04-07T04:02:28.56-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20209724, 20104862, 0 committed:false) SnapCreateDur 10.105µs SnapOpenDur 195.166516ms 2016-04-07T04:02:28.727-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20420000 2016-04-07T04:02:28.777-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20110000 QueuedCount 2998 2016-04-07T04:02:28.954-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20430000 2016-04-07T04:02:29.197-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20440000 2016-04-07T04:02:29.356-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.896µs CommitTime 421.324286ms TotalFlushTime 14m7.963225026s TotalCommitTime 1m7.608818975s 2016-04-07T04:02:29.356-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20222348, 20111174, 165 committed:true 2016-04-07T04:02:29.356-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20222348, 20111174, 165 committed:true) SnapCreateDur 432.424767ms SnapOpenDur 41.707µs 2016-04-07T04:02:29.38-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:29.403-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:29.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20450000 2016-04-07T04:02:29.711-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20120000 QueuedCount 12633 2016-04-07T04:02:29.717-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20460000 2016-04-07T04:02:29.95-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20470000 2016-04-07T04:02:30.244-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20480000 2016-04-07T04:02:30.278-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20130000 QueuedCount 2633 2016-04-07T04:02:30.458-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20260990, 20130495, 0 committed:false 2016-04-07T04:02:30.495-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20490000 2016-04-07T04:02:30.564-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20260990, 20130495, 0 committed:false) SnapCreateDur 11.941µs SnapOpenDur 106.124806ms 2016-04-07T04:02:30.588-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:30.612-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:30.696-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20500000 2016-04-07T04:02:30.924-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20510000 2016-04-07T04:02:31.004-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20140000 QueuedCount 15298 2016-04-07T04:02:31.249-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20520000 2016-04-07T04:02:31.465-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20530000 2016-04-07T04:02:31.656-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20150000 QueuedCount 5298 2016-04-07T04:02:31.68-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20540000 2016-04-07T04:02:31.93-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20550000 2016-04-07T04:02:31.987-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20305622, 20152811, 0 committed:false 2016-04-07T04:02:32.134-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20305622, 20152811, 0 committed:false) SnapCreateDur 14.172µs SnapOpenDur 147.473641ms 2016-04-07T04:02:32.158-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:32.183-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:32.227-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20560000 2016-04-07T04:02:32.43-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20570000 2016-04-07T04:02:32.44-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20160000 QueuedCount 18309 2016-04-07T04:02:32.648-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20580000 2016-04-07T04:02:32.888-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20590000 2016-04-07T04:02:33.175-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20600000 2016-04-07T04:02:33.39-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20610000 2016-04-07T04:02:33.591-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20620000 2016-04-07T04:02:33.834-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20630000 2016-04-07T04:02:33.898-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20170000 QueuedCount 8309 2016-04-07T04:02:34.092-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20640000 2016-04-07T04:02:34.316-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20650000 2016-04-07T04:02:34.388-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20350882, 20175441, 0 committed:false 2016-04-07T04:02:34.555-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20660000 2016-04-07T04:02:34.582-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20350882, 20175441, 0 committed:false) SnapCreateDur 10.78µs SnapOpenDur 194.354739ms 2016-04-07T04:02:34.699-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20180000 QueuedCount 5641 2016-04-07T04:02:34.759-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20670000 2016-04-07T04:02:34.993-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20680000 2016-04-07T04:02:35.287-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20690000 2016-04-07T04:02:35.405-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.148µs CommitTime 386.584377ms TotalFlushTime 14m13.02650238s TotalCommitTime 1m7.995403352s 2016-04-07T04:02:35.405-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20365256, 20182628, 166 committed:true 2016-04-07T04:02:35.405-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20365256, 20182628, 166 committed:true) SnapCreateDur 398.998046ms SnapOpenDur 36.239µs 2016-04-07T04:02:35.429-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:35.452-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:35.476-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:35.518-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20700000 2016-04-07T04:02:35.654-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20190000 QueuedCount 25997 2016-04-07T04:02:35.716-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20710000 2016-04-07T04:02:35.934-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20720000 2016-04-07T04:02:36.247-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20730000 2016-04-07T04:02:36.25-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20200000 QueuedCount 15997 2016-04-07T04:02:36.461-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20740000 2016-04-07T04:02:36.653-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20750000 2016-04-07T04:02:36.815-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20210000 QueuedCount 5997 2016-04-07T04:02:36.872-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20760000 2016-04-07T04:02:37.122-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20770000 2016-04-07T04:02:37.191-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20424984, 20212492, 0 committed:false 2016-04-07T04:02:37.344-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20424984, 20212492, 0 committed:false) SnapCreateDur 11.502µs SnapOpenDur 153.356542ms 2016-04-07T04:02:37.367-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:37.392-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:37.401-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20780000 2016-04-07T04:02:37.415-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:37.655-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20790000 2016-04-07T04:02:37.799-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20800000 2016-04-07T04:02:38.011-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20810000 2016-04-07T04:02:38.312-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20820000 2016-04-07T04:02:38.51-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20220000 QueuedCount 25251 2016-04-07T04:02:38.527-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20830000 2016-04-07T04:02:38.709-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20840000 2016-04-07T04:02:38.919-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20850000 2016-04-07T04:02:39.136-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20860000 2016-04-07T04:02:39.471-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20870000 2016-04-07T04:02:39.668-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20880000 2016-04-07T04:02:39.856-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20890000 2016-04-07T04:02:39.999-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20230000 QueuedCount 15251 2016-04-07T04:02:40.053-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20900000 2016-04-07T04:02:40.268-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20910000 2016-04-07T04:02:40.585-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20240000 QueuedCount 5251 2016-04-07T04:02:40.596-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20920000 2016-04-07T04:02:40.786-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20930000 2016-04-07T04:02:40.899-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20482586, 20241293, 0 committed:false 2016-04-07T04:02:40.987-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20940000 2016-04-07T04:02:41.094-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20482586, 20241293, 0 committed:false) SnapCreateDur 9.431µs SnapOpenDur 195.239702ms 2016-04-07T04:02:41.212-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20950000 2016-04-07T04:02:41.438-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20960000 2016-04-07T04:02:41.774-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20970000 2016-04-07T04:02:41.953-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20980000 2016-04-07T04:02:42.146-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 20990000 2016-04-07T04:02:42.302-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20250000 QueuedCount 718 2016-04-07T04:02:42.367-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21000000 2016-04-07T04:02:42.575-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21010000 2016-04-07T04:02:42.909-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21020000 2016-04-07T04:02:43.09-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21030000 2016-04-07T04:02:43.157-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.778µs CommitTime 397.482446ms TotalFlushTime 14m19.914114404s TotalCommitTime 1m8.392885798s 2016-04-07T04:02:43.157-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20493366, 20246683, 167 committed:true 2016-04-07T04:02:43.157-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20493366, 20246683, 167 committed:true) SnapCreateDur 409.443274ms SnapOpenDur 34.381µs 2016-04-07T04:02:43.181-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:43.205-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:43.229-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:43.299-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21040000 2016-04-07T04:02:43.517-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21050000 2016-04-07T04:02:43.763-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21060000 2016-04-07T04:02:43.863-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20260000 QueuedCount 19696 2016-04-07T04:02:44.082-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21070000 2016-04-07T04:02:44.273-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21080000 2016-04-07T04:02:44.482-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21090000 2016-04-07T04:02:44.506-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20270000 QueuedCount 9696 2016-04-07T04:02:44.702-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21100000 2016-04-07T04:02:44.956-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21110000 2016-04-07T04:02:45.238-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21120000 2016-04-07T04:02:45.457-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21130000 2016-04-07T04:02:45.654-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21140000 2016-04-07T04:02:45.858-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21150000 2016-04-07T04:02:46.043-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20550428, 20275214, 0 committed:false 2016-04-07T04:02:46.092-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21160000 2016-04-07T04:02:46.195-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20550428, 20275214, 0 committed:false) SnapCreateDur 13.137µs SnapOpenDur 152.379926ms 2016-04-07T04:02:46.222-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20280000 QueuedCount 8397 2016-04-07T04:02:46.352-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21170000 2016-04-07T04:02:46.622-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21180000 2016-04-07T04:02:46.836-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21190000 2016-04-07T04:02:47.034-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21200000 2016-04-07T04:02:47.103-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.42µs CommitTime 332.842311ms TotalFlushTime 14m23.294342235s TotalCommitTime 1m8.725728109s 2016-04-07T04:02:47.103-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20567580, 20283790, 168 committed:true 2016-04-07T04:02:47.103-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20567580, 20283790, 168 committed:true) SnapCreateDur 344.171404ms SnapOpenDur 39.736µs 2016-04-07T04:02:47.127-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:47.151-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:47.174-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:47.237-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20290000 QueuedCount 27867 2016-04-07T04:02:47.269-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21210000 2016-04-07T04:02:47.499-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21220000 2016-04-07T04:02:47.706-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21230000 2016-04-07T04:02:47.845-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20300000 QueuedCount 17867 2016-04-07T04:02:47.975-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21240000 2016-04-07T04:02:48.183-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21250000 2016-04-07T04:02:48.427-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21260000 2016-04-07T04:02:48.626-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21270000 2016-04-07T04:02:48.883-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21280000 2016-04-07T04:02:49.083-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21290000 2016-04-07T04:02:49.338-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20310000 QueuedCount 7867 2016-04-07T04:02:49.351-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21300000 2016-04-07T04:02:49.619-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21310000 2016-04-07T04:02:49.819-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21320000 2016-04-07T04:02:49.868-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20625648, 20312824, 0 committed:false 2016-04-07T04:02:50.02-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21330000 2016-04-07T04:02:50.031-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20625648, 20312824, 0 committed:false) SnapCreateDur 11.44µs SnapOpenDur 163.219584ms 2016-04-07T04:02:50.056-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:50.079-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:50.103-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:50.229-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20320000 QueuedCount 28338 2016-04-07T04:02:50.261-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21340000 2016-04-07T04:02:50.478-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21350000 2016-04-07T04:02:50.757-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21360000 2016-04-07T04:02:50.795-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20330000 QueuedCount 18338 2016-04-07T04:02:50.978-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21370000 2016-04-07T04:02:51.182-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21380000 2016-04-07T04:02:51.35-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20340000 QueuedCount 8338 2016-04-07T04:02:51.414-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21390000 2016-04-07T04:02:51.665-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21400000 2016-04-07T04:02:51.876-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21410000 2016-04-07T04:02:51.906-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20685638, 20342819, 0 committed:false 2016-04-07T04:02:52.124-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20685638, 20342819, 0 committed:false) SnapCreateDur 9.35µs SnapOpenDur 218.207795ms 2016-04-07T04:02:52.128-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21420000 2016-04-07T04:02:52.241-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20350000 QueuedCount 7704 2016-04-07T04:02:52.397-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21430000 2016-04-07T04:02:52.593-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21440000 2016-04-07T04:02:52.807-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21450000 2016-04-07T04:02:53.032-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21460000 2016-04-07T04:02:53.167-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.332µs CommitTime 455.366067ms TotalFlushTime 14m28.402024925s TotalCommitTime 1m9.181094176s 2016-04-07T04:02:53.167-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20704092, 20352046, 169 committed:true 2016-04-07T04:02:53.167-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20704092, 20352046, 169 committed:true) SnapCreateDur 466.785408ms SnapOpenDur 46.743µs 2016-04-07T04:02:53.191-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:53.215-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:53.238-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:53.24-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21470000 2016-04-07T04:02:53.313-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20360000 QueuedCount 28123 2016-04-07T04:02:53.49-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21480000 2016-04-07T04:02:53.793-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21490000 2016-04-07T04:02:53.998-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20370000 QueuedCount 18123 2016-04-07T04:02:54.013-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21500000 2016-04-07T04:02:54.205-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21510000 2016-04-07T04:02:54.447-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21520000 2016-04-07T04:02:54.661-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21530000 2016-04-07T04:02:54.675-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20380000 QueuedCount 8123 2016-04-07T04:02:54.892-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21540000 2016-04-07T04:02:55.151-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21550000 2016-04-07T04:02:55.173-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20764024, 20382012, 0 committed:false 2016-04-07T04:02:55.347-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20764024, 20382012, 0 committed:false) SnapCreateDur 11.472µs SnapOpenDur 173.858296ms 2016-04-07T04:02:55.373-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:55.397-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:55.407-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21560000 2016-04-07T04:02:55.42-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:55.48-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20390000 QueuedCount 30026 2016-04-07T04:02:55.609-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21570000 2016-04-07T04:02:55.848-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21580000 2016-04-07T04:02:56.047-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21590000 2016-04-07T04:02:56.121-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20400000 QueuedCount 20026 2016-04-07T04:02:56.272-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21600000 2016-04-07T04:02:56.486-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21610000 2016-04-07T04:02:56.69-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20410000 QueuedCount 10026 2016-04-07T04:02:56.736-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21620000 2016-04-07T04:02:57.004-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21630000 2016-04-07T04:02:57.195-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21640000 2016-04-07T04:02:57.332-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20420000 QueuedCount 26 2016-04-07T04:02:57.335-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20826736, 20413368, 0 committed:false 2016-04-07T04:02:57.435-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21650000 2016-04-07T04:02:57.576-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20826736, 20413368, 0 committed:false) SnapCreateDur 11.104µs SnapOpenDur 240.499468ms 2016-04-07T04:02:57.675-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21660000 2016-04-07T04:02:57.867-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21670000 2016-04-07T04:02:58.09-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21680000 2016-04-07T04:02:58.355-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21690000 2016-04-07T04:02:58.628-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21700000 2016-04-07T04:02:58.638-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.493µs CommitTime 464.969516ms TotalFlushTime 14m32.870403122s TotalCommitTime 1m9.646063692s 2016-04-07T04:02:58.639-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20844304, 20422152, 170 committed:true 2016-04-07T04:02:58.639-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20844304, 20422152, 170 committed:true) SnapCreateDur 476.36148ms SnapOpenDur 36.998µs 2016-04-07T04:02:58.663-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:58.679-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20430000 QueuedCount 7469 2016-04-07T04:02:58.687-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:58.71-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:02:58.858-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21710000 2016-04-07T04:02:59.058-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21720000 2016-04-07T04:02:59.26-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21730000 2016-04-07T04:02:59.458-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21740000 2016-04-07T04:02:59.461-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20440000 QueuedCount 20646 2016-04-07T04:02:59.679-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21750000 2016-04-07T04:02:59.93-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21760000 2016-04-07T04:03:00.205-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20450000 QueuedCount 10646 2016-04-07T04:03:00.212-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21770000 2016-04-07T04:03:00.457-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21780000 2016-04-07T04:03:00.676-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21790000 2016-04-07T04:03:00.852-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21800000 2016-04-07T04:03:00.87-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20460000 QueuedCount 646 2016-04-07T04:03:00.885-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 20906620, 20453310, 0 committed:false 2016-04-07T04:03:01.047-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21810000 2016-04-07T04:03:01.07-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 20906620, 20453310, 0 committed:false) SnapCreateDur 11.386µs SnapOpenDur 185.345118ms 2016-04-07T04:03:01.095-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:01.119-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:01.142-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:01.167-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:01.191-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:01.257-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21820000 2016-04-07T04:03:01.483-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21830000 2016-04-07T04:03:01.728-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20470000 QueuedCount 40180 2016-04-07T04:03:01.744-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21840000 2016-04-07T04:03:01.993-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21850000 2016-04-07T04:03:02.244-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21860000 2016-04-07T04:03:02.307-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20480000 QueuedCount 30180 2016-04-07T04:03:02.422-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21870000 2016-04-07T04:03:02.61-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21880000 2016-04-07T04:03:02.793-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21890000 2016-04-07T04:03:02.952-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20490000 QueuedCount 20180 2016-04-07T04:03:02.999-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21900000 2016-04-07T04:03:03.236-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21910000 2016-04-07T04:03:03.512-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21920000 2016-04-07T04:03:03.591-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20500000 QueuedCount 10180 2016-04-07T04:03:03.747-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21930000 2016-04-07T04:03:03.96-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21940000 2016-04-07T04:03:04.126-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21950000 2016-04-07T04:03:04.246-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20510000 QueuedCount 180 2016-04-07T04:03:04.256-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21004072, 20502036, 0 committed:false 2016-04-07T04:03:04.312-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21960000 2016-04-07T04:03:04.535-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21970000 2016-04-07T04:03:04.538-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21004072, 20502036, 0 committed:false) SnapCreateDur 15.364µs SnapOpenDur 281.757322ms 2016-04-07T04:03:04.745-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21980000 2016-04-07T04:03:05.022-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 21990000 2016-04-07T04:03:05.301-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22000000 2016-04-07T04:03:05.457-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.349µs CommitTime 524.281586ms TotalFlushTime 14m38.564273155s TotalCommitTime 1m10.170345278s 2016-04-07T04:03:05.457-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21014742, 20507371, 171 committed:true 2016-04-07T04:03:05.457-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21014742, 20507371, 171 committed:true) SnapCreateDur 535.792644ms SnapOpenDur 36.089µs 2016-04-07T04:03:05.481-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:05.505-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:05.523-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22010000 2016-04-07T04:03:05.53-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:05.554-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:05.578-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:05.72-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22020000 2016-04-07T04:03:05.722-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20520000 QueuedCount 43977 2016-04-07T04:03:05.901-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22030000 2016-04-07T04:03:06.081-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22040000 2016-04-07T04:03:06.263-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22050000 2016-04-07T04:03:06.417-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20530000 QueuedCount 33977 2016-04-07T04:03:06.512-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22060000 2016-04-07T04:03:06.751-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22070000 2016-04-07T04:03:07.043-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22080000 2016-04-07T04:03:07.093-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20540000 QueuedCount 23977 2016-04-07T04:03:07.273-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22090000 2016-04-07T04:03:07.467-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22100000 2016-04-07T04:03:07.65-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22110000 2016-04-07T04:03:07.738-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20550000 QueuedCount 13977 2016-04-07T04:03:07.847-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22120000 2016-04-07T04:03:08.079-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22130000 2016-04-07T04:03:08.292-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22140000 2016-04-07T04:03:08.325-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20560000 QueuedCount 3977 2016-04-07T04:03:08.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22150000 2016-04-07T04:03:08.601-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21109976, 20554988, 0 committed:false 2016-04-07T04:03:08.762-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22160000 2016-04-07T04:03:08.829-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21109976, 20554988, 0 committed:false) SnapCreateDur 11.229µs SnapOpenDur 227.639644ms 2016-04-07T04:03:08.854-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:08.878-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:08.902-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:08.925-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:08.949-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:09.068-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22170000 2016-04-07T04:03:09.241-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20570000 QueuedCount 46562 2016-04-07T04:03:09.263-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22180000 2016-04-07T04:03:09.454-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22190000 2016-04-07T04:03:09.646-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22200000 2016-04-07T04:03:09.871-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22210000 2016-04-07T04:03:09.892-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20580000 QueuedCount 36562 2016-04-07T04:03:10.065-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22220000 2016-04-07T04:03:10.287-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22230000 2016-04-07T04:03:10.524-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20590000 QueuedCount 26562 2016-04-07T04:03:10.548-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22240000 2016-04-07T04:03:10.817-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22250000 2016-04-07T04:03:11.07-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22260000 2016-04-07T04:03:11.255-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22270000 2016-04-07T04:03:11.474-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22280000 2016-04-07T04:03:11.678-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22290000 2016-04-07T04:03:11.881-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22300000 2016-04-07T04:03:12.106-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22310000 2016-04-07T04:03:12.112-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20600000 QueuedCount 16562 2016-04-07T04:03:12.333-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22320000 2016-04-07T04:03:12.598-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22330000 2016-04-07T04:03:12.693-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20610000 QueuedCount 6562 2016-04-07T04:03:12.894-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22340000 2016-04-07T04:03:12.947-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":1725636,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2599031079,"bucket-1:myindex:avg_ts_items_count":34872,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6676862676,"bucket-1:myindex:delete_bytes":24018252,"bucket-1:myindex:disk_size":19291815936,"bucket-1:myindex:flush_queue_size":3364,"bucket-1:myindex:frag_percent":65,"bucket-1:myindex:get_bytes":24407703,"bucket-1:myindex:insert_bytes":2158649115,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":171,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":20772406,"bucket-1:myindex:num_docs_pending":13352,"bucket-1:myindex:num_docs_queued":1656458,"bucket-1:myindex:num_flush_queued":20616562,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":190,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":3371926017,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"171 70170345278 -4544758520356487548","bucket-1:myindex:timings/storage_del":"603421 12450791280 653309488456262738","bucket-1:myindex:timings/storage_get":"20613199 134053093682 613734504548189958","bucket-1:myindex:timings/storage_handle_open":"171 4170934 106609441386","bucket-1:myindex:timings/storage_info":"723 6714630 172522242606","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"174 148576230 3666144681370324","bucket-1:myindex:timings/storage_meta_set":"171 395522075 107842441231125499","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"41206841 725771025743 5133349277368724095","bucket-1:myindex:timings/storage_snapshot_close":"18 399755883 9966327386931869","bucket-1:myindex:timings/storage_snapshot_create":"19 8625414573 -5690301615538806969","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":22342198,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":196,"memory_quota":31457280000,"memory_used":6982209536,"needs_restart":false,"num_connections":0,"timings/stats_response":"3787 526431370 90119677820904"} 2016-04-07T04:03:13.108-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22350000 2016-04-07T04:03:13.155-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21213448, 20606724, 0 committed:false 2016-04-07T04:03:13.326-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22360000 2016-04-07T04:03:13.461-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21213448, 20606724, 0 committed:false) SnapCreateDur 15.25µs SnapOpenDur 306.342038ms 2016-04-07T04:03:13.506-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22370000 2016-04-07T04:03:13.676-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20620000 QueuedCount 4851 2016-04-07T04:03:13.748-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22380000 2016-04-07T04:03:13.973-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22390000 2016-04-07T04:03:14.192-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22400000 2016-04-07T04:03:14.392-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22410000 2016-04-07T04:03:14.582-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.653µs CommitTime 614.579802ms TotalFlushTime 14m46.401683743s TotalCommitTime 1m10.78492508s 2016-04-07T04:03:14.582-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21229730, 20614865, 172 committed:true 2016-04-07T04:03:14.582-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21229730, 20614865, 172 committed:true) SnapCreateDur 626.634467ms SnapOpenDur 37.355µs 2016-04-07T04:03:14.607-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:14.631-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:14.655-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:14.679-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:14.703-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:14.704-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22420000 2016-04-07T04:03:14.951-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20630000 QueuedCount 47319 2016-04-07T04:03:14.989-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22430000 2016-04-07T04:03:15.225-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22440000 2016-04-07T04:03:15.412-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22450000 2016-04-07T04:03:15.646-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22460000 2016-04-07T04:03:15.736-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20640000 QueuedCount 37319 2016-04-07T04:03:15.803-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22470000 2016-04-07T04:03:16.015-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22480000 2016-04-07T04:03:16.236-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22490000 2016-04-07T04:03:16.445-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20650000 QueuedCount 27319 2016-04-07T04:03:16.465-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22500000 2016-04-07T04:03:16.825-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22510000 2016-04-07T04:03:17.039-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22520000 2016-04-07T04:03:17.096-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20660000 QueuedCount 17319 2016-04-07T04:03:17.294-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22530000 2016-04-07T04:03:17.485-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22540000 2016-04-07T04:03:17.664-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22550000 2016-04-07T04:03:17.755-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20670000 QueuedCount 7319 2016-04-07T04:03:17.865-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22560000 2016-04-07T04:03:18.094-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22570000 2016-04-07T04:03:18.227-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21333008, 20666504, 0 committed:false 2016-04-07T04:03:18.322-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22580000 2016-04-07T04:03:18.472-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21333008, 20666504, 0 committed:false) SnapCreateDur 12.483µs SnapOpenDur 245.51518ms 2016-04-07T04:03:18.545-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22590000 2016-04-07T04:03:18.621-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20680000 QueuedCount 2299 2016-04-07T04:03:18.876-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22600000 2016-04-07T04:03:19.116-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22610000 2016-04-07T04:03:19.35-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22620000 2016-04-07T04:03:19.552-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22630000 2016-04-07T04:03:19.737-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22640000 2016-04-07T04:03:19.837-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.202µs CommitTime 489.803336ms TotalFlushTime 14m50.827213663s TotalCommitTime 1m11.274728416s 2016-04-07T04:03:19.837-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21342806, 20671403, 173 committed:true 2016-04-07T04:03:19.837-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21342806, 20671403, 173 committed:true) SnapCreateDur 501.486331ms SnapOpenDur 45.932µs 2016-04-07T04:03:19.862-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:19.886-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:19.911-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:19.935-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:19.946-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22650000 2016-04-07T04:03:19.958-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:20.152-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22660000 2016-04-07T04:03:20.377-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22670000 2016-04-07T04:03:20.398-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20690000 QueuedCount 43657 2016-04-07T04:03:20.631-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22680000 2016-04-07T04:03:20.86-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22690000 2016-04-07T04:03:21.076-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20700000 QueuedCount 33657 2016-04-07T04:03:21.192-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22700000 2016-04-07T04:03:21.404-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22710000 2016-04-07T04:03:21.638-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22720000 2016-04-07T04:03:21.738-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20710000 QueuedCount 23657 2016-04-07T04:03:21.842-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22730000 2016-04-07T04:03:22.029-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22740000 2016-04-07T04:03:22.241-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22750000 2016-04-07T04:03:22.39-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20720000 QueuedCount 13657 2016-04-07T04:03:22.448-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22760000 2016-04-07T04:03:22.7-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22770000 2016-04-07T04:03:22.906-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22780000 2016-04-07T04:03:23.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20730000 QueuedCount 3657 2016-04-07T04:03:23.109-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22790000 2016-04-07T04:03:23.253-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21443954, 20721977, 0 committed:false 2016-04-07T04:03:23.442-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22800000 2016-04-07T04:03:23.496-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21443954, 20721977, 0 committed:false) SnapCreateDur 10.978µs SnapOpenDur 243.797508ms 2016-04-07T04:03:23.644-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22810000 2016-04-07T04:03:23.871-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22820000 2016-04-07T04:03:23.987-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20740000 QueuedCount 2723 2016-04-07T04:03:24.061-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22830000 2016-04-07T04:03:24.256-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22840000 2016-04-07T04:03:24.483-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22850000 2016-04-07T04:03:24.685-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.776µs CommitTime 490.87643ms TotalFlushTime 14m54.841491755s TotalCommitTime 1m11.765604846s 2016-04-07T04:03:24.685-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21461834, 20730917, 174 committed:true 2016-04-07T04:03:24.685-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21461834, 20730917, 174 committed:true) SnapCreateDur 503.417982ms SnapOpenDur 41.044µs 2016-04-07T04:03:24.697-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22860000 2016-04-07T04:03:24.71-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:24.734-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:24.759-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:24.782-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:24.806-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:24.917-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22870000 2016-04-07T04:03:25.132-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22880000 2016-04-07T04:03:25.248-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20750000 QueuedCount 45440 2016-04-07T04:03:25.353-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22890000 2016-04-07T04:03:25.661-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22900000 2016-04-07T04:03:25.886-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22910000 2016-04-07T04:03:25.931-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20760000 QueuedCount 35440 2016-04-07T04:03:26.109-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22920000 2016-04-07T04:03:26.318-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22930000 2016-04-07T04:03:26.531-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22940000 2016-04-07T04:03:26.619-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20770000 QueuedCount 25440 2016-04-07T04:03:26.785-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22950000 2016-04-07T04:03:26.992-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22960000 2016-04-07T04:03:27.199-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22970000 2016-04-07T04:03:27.274-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20780000 QueuedCount 15440 2016-04-07T04:03:27.414-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22980000 2016-04-07T04:03:27.63-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 22990000 2016-04-07T04:03:27.907-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23000000 2016-04-07T04:03:27.93-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20790000 QueuedCount 5440 2016-04-07T04:03:28.171-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23010000 2016-04-07T04:03:28.224-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21565586, 20782793, 0 committed:false 2016-04-07T04:03:28.38-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23020000 2016-04-07T04:03:28.493-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21565586, 20782793, 0 committed:false) SnapCreateDur 15.808µs SnapOpenDur 269.472733ms 2016-04-07T04:03:28.518-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:28.633-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23030000 2016-04-07T04:03:28.857-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23040000 2016-04-07T04:03:28.882-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20800000 QueuedCount 14122 2016-04-07T04:03:29.058-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23050000 2016-04-07T04:03:29.246-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23060000 2016-04-07T04:03:29.453-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20810000 QueuedCount 4122 2016-04-07T04:03:29.461-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23070000 2016-04-07T04:03:29.811-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23080000 2016-04-07T04:03:29.915-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23090000 2016-04-07T04:03:30.149-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23100000 2016-04-07T04:03:30.425-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.781µs CommitTime 580.228843ms TotalFlushTime 14m59.640893179s TotalCommitTime 1m12.345833689s 2016-04-07T04:03:30.425-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21602376, 20801188, 175 committed:true 2016-04-07T04:03:30.425-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21602376, 20801188, 175 committed:true) SnapCreateDur 592.969985ms SnapOpenDur 42.172µs 2016-04-07T04:03:30.444-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23110000 2016-04-07T04:03:30.45-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:30.477-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:30.502-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:30.526-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:30.55-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:30.693-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23120000 2016-04-07T04:03:30.936-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23130000 2016-04-07T04:03:31.189-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23140000 2016-04-07T04:03:31.402-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23150000 2016-04-07T04:03:31.623-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23160000 2016-04-07T04:03:31.722-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20820000 QueuedCount 44481 2016-04-07T04:03:31.841-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23170000 2016-04-07T04:03:32.06-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23180000 2016-04-07T04:03:32.28-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23190000 2016-04-07T04:03:32.452-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20830000 QueuedCount 34481 2016-04-07T04:03:32.509-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23200000 2016-04-07T04:03:32.746-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23210000 2016-04-07T04:03:33.05-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23220000 2016-04-07T04:03:33.161-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20840000 QueuedCount 24481 2016-04-07T04:03:33.309-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23230000 2016-04-07T04:03:33.528-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23240000 2016-04-07T04:03:33.75-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23250000 2016-04-07T04:03:33.856-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20850000 QueuedCount 14481 2016-04-07T04:03:33.962-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23260000 2016-04-07T04:03:34.172-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23270000 2016-04-07T04:03:34.362-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23280000 2016-04-07T04:03:34.57-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20860000 QueuedCount 4481 2016-04-07T04:03:34.579-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23290000 2016-04-07T04:03:34.801-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23300000 2016-04-07T04:03:34.882-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21701456, 20850728, 0 committed:false 2016-04-07T04:03:35.037-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23310000 2016-04-07T04:03:35.145-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21701456, 20850728, 0 committed:false) SnapCreateDur 11.84µs SnapOpenDur 263.253159ms 2016-04-07T04:03:35.286-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23320000 2016-04-07T04:03:35.54-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20870000 QueuedCount 1773 2016-04-07T04:03:35.622-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23330000 2016-04-07T04:03:35.856-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23340000 2016-04-07T04:03:36.053-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23350000 2016-04-07T04:03:36.267-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23360000 2016-04-07T04:03:36.315-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.018µs CommitTime 663.837517ms TotalFlushTime 15m4.515491825s TotalCommitTime 1m13.009671206s 2016-04-07T04:03:36.315-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21715818, 20857909, 176 committed:true 2016-04-07T04:03:36.315-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21715818, 20857909, 176 committed:true) SnapCreateDur 675.655814ms SnapOpenDur 34.596µs 2016-04-07T04:03:36.34-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:36.365-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:36.39-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:36.414-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:36.437-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:36.49-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23370000 2016-04-07T04:03:36.723-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23380000 2016-04-07T04:03:36.911-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23390000 2016-04-07T04:03:37.111-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23400000 2016-04-07T04:03:37.336-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23410000 2016-04-07T04:03:37.569-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23420000 2016-04-07T04:03:37.825-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23430000 2016-04-07T04:03:37.899-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20880000 QueuedCount 46210 2016-04-07T04:03:38.133-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23440000 2016-04-07T04:03:38.415-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23450000 2016-04-07T04:03:38.606-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23460000 2016-04-07T04:03:38.609-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20890000 QueuedCount 36210 2016-04-07T04:03:38.842-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23470000 2016-04-07T04:03:39.079-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23480000 2016-04-07T04:03:39.289-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23490000 2016-04-07T04:03:39.311-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20900000 QueuedCount 26210 2016-04-07T04:03:39.504-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23500000 2016-04-07T04:03:39.697-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23510000 2016-04-07T04:03:39.905-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23520000 2016-04-07T04:03:39.975-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20910000 QueuedCount 16210 2016-04-07T04:03:40.13-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23530000 2016-04-07T04:03:40.38-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23540000 2016-04-07T04:03:40.605-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23550000 2016-04-07T04:03:40.879-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23560000 2016-04-07T04:03:41.149-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23570000 2016-04-07T04:03:41.363-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23580000 2016-04-07T04:03:41.561-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23590000 2016-04-07T04:03:41.655-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20920000 QueuedCount 6210 2016-04-07T04:03:41.83-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23600000 2016-04-07T04:03:42.027-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23610000 2016-04-07T04:03:42.268-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23620000 2016-04-07T04:03:42.457-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23630000 2016-04-07T04:03:42.715-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23640000 2016-04-07T04:03:42.958-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23650000 2016-04-07T04:03:43.175-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23660000 2016-04-07T04:03:43.396-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23670000 2016-04-07T04:03:43.555-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21822972, 20911486, 0 committed:false 2016-04-07T04:03:43.69-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23680000 2016-04-07T04:03:43.836-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21822972, 20911486, 0 committed:false) SnapCreateDur 10.158µs SnapOpenDur 281.262263ms 2016-04-07T04:03:43.974-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23690000 2016-04-07T04:03:44.173-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23700000 2016-04-07T04:03:44.356-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23710000 2016-04-07T04:03:44.489-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20930000 QueuedCount 7130 2016-04-07T04:03:44.611-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23720000 2016-04-07T04:03:44.846-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23730000 2016-04-07T04:03:45.05-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23740000 2016-04-07T04:03:45.288-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23750000 2016-04-07T04:03:45.497-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23760000 2016-04-07T04:03:45.711-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23770000 2016-04-07T04:03:45.945-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23780000 2016-04-07T04:03:46.167-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23790000 2016-04-07T04:03:46.394-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23800000 2016-04-07T04:03:46.41-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.55µs CommitTime 556.008031ms TotalFlushTime 15m13.671455852s TotalCommitTime 1m13.565679237s 2016-04-07T04:03:46.411-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21844452, 20922226, 177 committed:true 2016-04-07T04:03:46.411-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21844452, 20922226, 177 committed:true) SnapCreateDur 569.034302ms SnapOpenDur 33.578µs 2016-04-07T04:03:46.436-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:46.461-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:46.485-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:46.508-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:46.534-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:46.75-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23810000 2016-04-07T04:03:46.829-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20940000 QueuedCount 48994 2016-04-07T04:03:46.955-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23820000 2016-04-07T04:03:47.167-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23830000 2016-04-07T04:03:47.392-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23840000 2016-04-07T04:03:47.579-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20950000 QueuedCount 38994 2016-04-07T04:03:47.627-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23850000 2016-04-07T04:03:47.857-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23860000 2016-04-07T04:03:48.06-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23870000 2016-04-07T04:03:48.26-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23880000 2016-04-07T04:03:48.455-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23890000 2016-04-07T04:03:48.651-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23900000 2016-04-07T04:03:48.87-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23910000 2016-04-07T04:03:49.006-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20960000 QueuedCount 28994 2016-04-07T04:03:49.092-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23920000 2016-04-07T04:03:49.311-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23930000 2016-04-07T04:03:49.629-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23940000 2016-04-07T04:03:49.873-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20970000 QueuedCount 18994 2016-04-07T04:03:49.879-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23950000 2016-04-07T04:03:50.076-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23960000 2016-04-07T04:03:50.307-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23970000 2016-04-07T04:03:50.544-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20980000 QueuedCount 8994 2016-04-07T04:03:50.561-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23980000 2016-04-07T04:03:50.75-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 23990000 2016-04-07T04:03:50.968-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24000000 2016-04-07T04:03:51.124-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21946512, 20973256, 0 committed:false 2016-04-07T04:03:51.168-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24010000 2016-04-07T04:03:51.371-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24020000 2016-04-07T04:03:51.393-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21946512, 20973256, 0 committed:false) SnapCreateDur 11.626µs SnapOpenDur 268.707882ms 2016-04-07T04:03:51.503-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 20990000 QueuedCount 8363 2016-04-07T04:03:51.593-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24030000 2016-04-07T04:03:51.814-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24040000 2016-04-07T04:03:52.027-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24050000 2016-04-07T04:03:52.247-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24060000 2016-04-07T04:03:52.524-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24070000 2016-04-07T04:03:52.605-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.864µs CommitTime 516.678369ms TotalFlushTime 15m18.990405756s TotalCommitTime 1m14.082357606s 2016-04-07T04:03:52.605-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 21964938, 20982469, 178 committed:true 2016-04-07T04:03:52.605-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 21964938, 20982469, 178 committed:true) SnapCreateDur 528.12943ms SnapOpenDur 42.516µs 2016-04-07T04:03:52.63-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:52.652-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:52.677-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:52.701-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:52.726-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:52.745-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21000000 QueuedCount 41161 2016-04-07T04:03:52.83-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24080000 2016-04-07T04:03:53.076-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24090000 2016-04-07T04:03:53.303-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24100000 2016-04-07T04:03:53.505-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24110000 2016-04-07T04:03:53.7-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24120000 2016-04-07T04:03:53.886-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24130000 2016-04-07T04:03:54.101-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24140000 2016-04-07T04:03:54.304-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24150000 2016-04-07T04:03:54.333-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21010000 QueuedCount 40417 2016-04-07T04:03:54.524-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24160000 2016-04-07T04:03:54.735-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24170000 2016-04-07T04:03:54.964-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24180000 2016-04-07T04:03:55.192-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24190000 2016-04-07T04:03:55.426-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24200000 2016-04-07T04:03:55.839-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24210000 2016-04-07T04:03:55.951-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21020000 QueuedCount 30417 2016-04-07T04:03:56.052-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24220000 2016-04-07T04:03:56.206-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24230000 2016-04-07T04:03:56.431-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24240000 2016-04-07T04:03:56.616-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21030000 QueuedCount 20417 2016-04-07T04:03:56.682-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24250000 2016-04-07T04:03:56.843-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24260000 2016-04-07T04:03:57.038-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24270000 2016-04-07T04:03:57.22-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24280000 2016-04-07T04:03:57.276-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21040000 QueuedCount 10417 2016-04-07T04:03:57.426-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24290000 2016-04-07T04:03:57.63-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24300000 2016-04-07T04:03:57.826-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24310000 2016-04-07T04:03:57.94-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21050000 QueuedCount 417 2016-04-07T04:03:57.953-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22067468, 21033734, 0 committed:false 2016-04-07T04:03:58.037-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24320000 2016-04-07T04:03:58.26-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22067468, 21033734, 0 committed:false) SnapCreateDur 16.947µs SnapOpenDur 306.306068ms 2016-04-07T04:03:58.264-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24330000 2016-04-07T04:03:58.515-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24340000 2016-04-07T04:03:58.807-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24350000 2016-04-07T04:03:59.042-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24360000 2016-04-07T04:03:59.222-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24370000 2016-04-07T04:03:59.393-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24380000 2016-04-07T04:03:59.415-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.65µs CommitTime 559.279566ms TotalFlushTime 15m24.838839137s TotalCommitTime 1m14.641637172s 2016-04-07T04:03:59.415-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22083740, 21041870, 179 committed:true 2016-04-07T04:03:59.415-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22083740, 21041870, 179 committed:true) SnapCreateDur 570.82798ms SnapOpenDur 39.667µs 2016-04-07T04:03:59.438-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:59.458-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21060000 QueuedCount 3682 2016-04-07T04:03:59.462-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:59.487-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:59.511-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:59.535-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:03:59.638-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24390000 2016-04-07T04:03:59.828-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24400000 2016-04-07T04:04:00.003-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24410000 2016-04-07T04:04:00.205-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24420000 2016-04-07T04:04:00.336-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21070000 QueuedCount 39023 2016-04-07T04:04:00.393-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24430000 2016-04-07T04:04:00.6-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24440000 2016-04-07T04:04:00.849-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24450000 2016-04-07T04:04:01.088-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21080000 QueuedCount 29023 2016-04-07T04:04:01.093-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24460000 2016-04-07T04:04:01.294-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24470000 2016-04-07T04:04:01.497-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24480000 2016-04-07T04:04:01.757-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24490000 2016-04-07T04:04:01.826-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21090000 QueuedCount 19023 2016-04-07T04:04:02.052-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24500000 2016-04-07T04:04:02.283-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24510000 2016-04-07T04:04:02.487-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24520000 2016-04-07T04:04:02.536-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21100000 QueuedCount 9023 2016-04-07T04:04:02.714-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24530000 2016-04-07T04:04:02.921-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24540000 2016-04-07T04:04:03.086-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24550000 2016-04-07T04:04:03.169-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22182812, 21091406, 0 committed:false 2016-04-07T04:04:03.258-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24560000 2016-04-07T04:04:03.438-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24570000 2016-04-07T04:04:03.477-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22182812, 21091406, 0 committed:false) SnapCreateDur 12.458µs SnapOpenDur 308.600633ms 2016-04-07T04:04:03.585-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21110000 QueuedCount 5959 2016-04-07T04:04:03.69-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24580000 2016-04-07T04:04:03.909-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24590000 2016-04-07T04:04:04.113-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24600000 2016-04-07T04:04:04.331-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24610000 2016-04-07T04:04:04.546-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.468µs CommitTime 571.906585ms TotalFlushTime 15m29.000269144s TotalCommitTime 1m15.213543757s 2016-04-07T04:04:04.546-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22196464, 21098232, 180 committed:true 2016-04-07T04:04:04.546-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22196464, 21098232, 180 committed:true) SnapCreateDur 584.416413ms SnapOpenDur 39.011µs 2016-04-07T04:04:04.568-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24620000 2016-04-07T04:04:04.57-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:04.595-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:04.62-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:04.644-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:04.668-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:04.787-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24630000 2016-04-07T04:04:04.835-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21120000 QueuedCount 48928 2016-04-07T04:04:05.045-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24640000 2016-04-07T04:04:05.348-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24650000 2016-04-07T04:04:05.588-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24660000 2016-04-07T04:04:05.623-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21130000 QueuedCount 38928 2016-04-07T04:04:05.791-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24670000 2016-04-07T04:04:06.043-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24680000 2016-04-07T04:04:06.254-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24690000 2016-04-07T04:04:06.38-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21140000 QueuedCount 28928 2016-04-07T04:04:06.444-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24700000 2016-04-07T04:04:06.671-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24710000 2016-04-07T04:04:07.02-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24720000 2016-04-07T04:04:07.088-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24730000 2016-04-07T04:04:07.262-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24740000 2016-04-07T04:04:07.374-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21150000 QueuedCount 18928 2016-04-07T04:04:07.456-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24750000 2016-04-07T04:04:07.654-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24760000 2016-04-07T04:04:07.884-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24770000 2016-04-07T04:04:08.107-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24780000 2016-04-07T04:04:08.241-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21160000 QueuedCount 8928 2016-04-07T04:04:08.325-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24790000 2016-04-07T04:04:08.62-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24800000 2016-04-07T04:04:08.867-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22300836, 21150418, 0 committed:false 2016-04-07T04:04:08.922-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24810000 2016-04-07T04:04:09.111-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24820000 2016-04-07T04:04:09.163-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22300836, 21150418, 0 committed:false) SnapCreateDur 14.08µs SnapOpenDur 296.43707ms 2016-04-07T04:04:09.282-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21170000 QueuedCount 6585 2016-04-07T04:04:09.332-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24830000 2016-04-07T04:04:09.617-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24840000 2016-04-07T04:04:09.835-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24850000 2016-04-07T04:04:10.011-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24860000 2016-04-07T04:04:10.206-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24870000 2016-04-07T04:04:10.31-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.462µs CommitTime 590.343583ms TotalFlushTime 15m33.771943493s TotalCommitTime 1m15.80388734s 2016-04-07T04:04:10.31-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22315924, 21157962, 181 committed:true 2016-04-07T04:04:10.31-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22315924, 21157962, 181 committed:true) SnapCreateDur 603.384786ms SnapOpenDur 37.65µs 2016-04-07T04:04:10.334-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:10.359-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:10.384-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:10.407-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24880000 2016-04-07T04:04:10.408-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:10.432-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:10.587-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21180000 QueuedCount 50000 2016-04-07T04:04:10.597-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24890000 2016-04-07T04:04:10.788-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24900000 2016-04-07T04:04:10.993-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24910000 2016-04-07T04:04:11.211-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24920000 2016-04-07T04:04:11.318-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21190000 QueuedCount 40756 2016-04-07T04:04:11.43-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24930000 2016-04-07T04:04:11.649-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24940000 2016-04-07T04:04:11.874-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24950000 2016-04-07T04:04:12.171-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24960000 2016-04-07T04:04:12.481-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24970000 2016-04-07T04:04:12.713-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21200000 QueuedCount 30756 2016-04-07T04:04:12.735-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24980000 2016-04-07T04:04:12.928-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 24990000 2016-04-07T04:04:12.947-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":3760480,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2283546853,"bucket-1:myindex:avg_ts_items_count":22527,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6785337863,"bucket-1:myindex:delete_bytes":49557543,"bucket-1:myindex:disk_size":22609399808,"bucket-1:myindex:flush_queue_size":30452,"bucket-1:myindex:frag_percent":69,"bucket-1:myindex:get_bytes":50354720,"bucket-1:myindex:insert_bytes":2221012562,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":181,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":21380793,"bucket-1:myindex:num_docs_pending":11580,"bucket-1:myindex:num_docs_queued":3845500,"bucket-1:myindex:num_flush_queued":21230756,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":210,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1147369689,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"181 75803887340 -1344359662188225018","bucket-1:myindex:timings/storage_del":"1181305 28137725498 1564778993591906434","bucket-1:myindex:timings/storage_get":"21200305 139854120313 640138271235042661","bucket-1:myindex:timings/storage_handle_open":"181 4385310 111273621240","bucket-1:myindex:timings/storage_info":"783 7101376 178687605122","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"184 152617277 3667887490835259","bucket-1:myindex:timings/storage_meta_set":"181 400037609 107844580711706541","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"42362610 754589054183 6689345984918646073","bucket-1:myindex:timings/storage_snapshot_close":"29 788963506 23849425034786896","bucket-1:myindex:timings/storage_snapshot_create":"29 11414409598 -4906967939490388482","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":24991236,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":425,"memory_quota":31457280000,"memory_used":8865288192,"needs_restart":false,"num_connections":0,"timings/stats_response":"3870 541796788 93060292443448"} 2016-04-07T04:04:13.162-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25000000 2016-04-07T04:04:13.391-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25010000 2016-04-07T04:04:13.58-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25020000 2016-04-07T04:04:13.737-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21210000 QueuedCount 20756 2016-04-07T04:04:13.788-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25030000 2016-04-07T04:04:13.984-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25040000 2016-04-07T04:04:14.181-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25050000 2016-04-07T04:04:14.381-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25060000 2016-04-07T04:04:14.463-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21220000 QueuedCount 10756 2016-04-07T04:04:14.581-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25070000 2016-04-07T04:04:14.786-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25080000 2016-04-07T04:04:15-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25090000 2016-04-07T04:04:15.171-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21230000 QueuedCount 756 2016-04-07T04:04:15.196-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25100000 2016-04-07T04:04:15.264-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22422530, 21211265, 0 committed:false 2016-04-07T04:04:15.424-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25110000 2016-04-07T04:04:15.56-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22422530, 21211265, 0 committed:false) SnapCreateDur 12.029µs SnapOpenDur 295.783862ms 2016-04-07T04:04:15.807-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25120000 2016-04-07T04:04:16.063-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25130000 2016-04-07T04:04:16.248-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25140000 2016-04-07T04:04:16.423-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25150000 2016-04-07T04:04:16.62-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25160000 2016-04-07T04:04:16.629-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.07µs CommitTime 554.147911ms TotalFlushTime 15m39.140251977s TotalCommitTime 1m16.358035251s 2016-04-07T04:04:16.629-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22436222, 21218111, 182 committed:true 2016-04-07T04:04:16.629-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22436222, 21218111, 182 committed:true) SnapCreateDur 565.528585ms SnapOpenDur 36.753µs 2016-04-07T04:04:16.654-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:16.678-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:16.702-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:16.726-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:16.752-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:16.786-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21240000 QueuedCount 50000 2016-04-07T04:04:16.828-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25170000 2016-04-07T04:04:17.02-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25180000 2016-04-07T04:04:17.211-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25190000 2016-04-07T04:04:17.402-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25200000 2016-04-07T04:04:17.556-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21250000 QueuedCount 42000 2016-04-07T04:04:17.599-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25210000 2016-04-07T04:04:17.806-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25220000 2016-04-07T04:04:17.994-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25230000 2016-04-07T04:04:18.191-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25240000 2016-04-07T04:04:18.266-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21260000 QueuedCount 32000 2016-04-07T04:04:18.426-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25250000 2016-04-07T04:04:18.657-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25260000 2016-04-07T04:04:18.811-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25270000 2016-04-07T04:04:18.949-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25280000 2016-04-07T04:04:19.161-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25290000 2016-04-07T04:04:19.53-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25300000 2016-04-07T04:04:19.719-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25310000 2016-04-07T04:04:19.917-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25320000 2016-04-07T04:04:20-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21270000 QueuedCount 22000 2016-04-07T04:04:20.106-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25330000 2016-04-07T04:04:20.344-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25340000 2016-04-07T04:04:20.559-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25350000 2016-04-07T04:04:20.733-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25360000 2016-04-07T04:04:20.742-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21280000 QueuedCount 12000 2016-04-07T04:04:20.934-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25370000 2016-04-07T04:04:21.198-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25380000 2016-04-07T04:04:21.387-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25390000 2016-04-07T04:04:21.565-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21290000 QueuedCount 2000 2016-04-07T04:04:21.603-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25400000 2016-04-07T04:04:21.689-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22543032, 21271516, 0 committed:false 2016-04-07T04:04:21.807-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25410000 2016-04-07T04:04:22.01-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22543032, 21271516, 0 committed:false) SnapCreateDur 13.249µs SnapOpenDur 320.787266ms 2016-04-07T04:04:22.012-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25420000 2016-04-07T04:04:22.198-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25430000 2016-04-07T04:04:22.404-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25440000 2016-04-07T04:04:22.596-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25450000 2016-04-07T04:04:22.803-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25460000 2016-04-07T04:04:23.112-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.997µs CommitTime 586.411179ms TotalFlushTime 15m44.619191608s TotalCommitTime 1m16.94444643s 2016-04-07T04:04:23.112-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22557162, 21278581, 183 committed:true 2016-04-07T04:04:23.112-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22557162, 21278581, 183 committed:true) SnapCreateDur 598.114684ms SnapOpenDur 33.681µs 2016-04-07T04:04:23.136-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:23.148-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21300000 QueuedCount 5715 2016-04-07T04:04:23.161-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:23.184-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25470000 2016-04-07T04:04:23.185-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:23.209-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:23.233-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:23.419-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25480000 2016-04-07T04:04:23.605-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25490000 2016-04-07T04:04:23.819-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25500000 2016-04-07T04:04:23.908-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21310000 QueuedCount 42183 2016-04-07T04:04:24.008-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25510000 2016-04-07T04:04:24.305-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25520000 2016-04-07T04:04:24.545-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25530000 2016-04-07T04:04:24.615-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21320000 QueuedCount 32183 2016-04-07T04:04:24.673-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25540000 2016-04-07T04:04:24.874-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25550000 2016-04-07T04:04:24.993-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25560000 2016-04-07T04:04:25.135-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25570000 2016-04-07T04:04:25.317-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25580000 2016-04-07T04:04:25.486-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25590000 2016-04-07T04:04:25.666-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25600000 2016-04-07T04:04:25.85-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25610000 2016-04-07T04:04:26.032-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25620000 2016-04-07T04:04:26.222-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25630000 2016-04-07T04:04:26.322-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21330000 QueuedCount 22183 2016-04-07T04:04:26.436-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25640000 2016-04-07T04:04:26.759-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25650000 2016-04-07T04:04:26.988-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21340000 QueuedCount 12183 2016-04-07T04:04:27.102-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25660000 2016-04-07T04:04:27.286-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25670000 2016-04-07T04:04:27.464-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25680000 2016-04-07T04:04:27.667-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25690000 2016-04-07T04:04:27.806-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21350000 QueuedCount 2183 2016-04-07T04:04:27.863-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25700000 2016-04-07T04:04:28.12-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25710000 2016-04-07T04:04:28.353-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25720000 2016-04-07T04:04:28.518-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25730000 2016-04-07T04:04:28.722-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25740000 2016-04-07T04:04:28.815-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22661380, 21330690, 0 committed:false 2016-04-07T04:04:28.906-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25750000 2016-04-07T04:04:29.086-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25760000 2016-04-07T04:04:29.123-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22661380, 21330690, 0 committed:false) SnapCreateDur 11.578µs SnapOpenDur 308.465245ms 2016-04-07T04:04:29.297-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25770000 2016-04-07T04:04:29.506-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25780000 2016-04-07T04:04:29.686-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25790000 2016-04-07T04:04:29.894-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25800000 2016-04-07T04:04:30.092-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25810000 2016-04-07T04:04:30.198-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.873µs CommitTime 569.093552ms TotalFlushTime 15m50.735107518s TotalCommitTime 1m17.513539982s 2016-04-07T04:04:30.198-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22674198, 21337099, 184 committed:true 2016-04-07T04:04:30.198-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22674198, 21337099, 184 committed:true) SnapCreateDur 580.997046ms SnapOpenDur 40.978µs 2016-04-07T04:04:30.222-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:30.242-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21360000 QueuedCount 6956 2016-04-07T04:04:30.248-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:30.272-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:30.545-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:30.55-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25820000 2016-04-07T04:04:30.569-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:30.746-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25830000 2016-04-07T04:04:30.948-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25840000 2016-04-07T04:04:31.122-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25850000 2016-04-07T04:04:31.288-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25860000 2016-04-07T04:04:31.29-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21370000 QueuedCount 42157 2016-04-07T04:04:31.48-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25870000 2016-04-07T04:04:31.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25880000 2016-04-07T04:04:31.886-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25890000 2016-04-07T04:04:32.018-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21380000 QueuedCount 32157 2016-04-07T04:04:32.156-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25900000 2016-04-07T04:04:32.352-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25910000 2016-04-07T04:04:32.546-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25920000 2016-04-07T04:04:32.691-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21390000 QueuedCount 22157 2016-04-07T04:04:32.735-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25930000 2016-04-07T04:04:32.927-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25940000 2016-04-07T04:04:33.134-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25950000 2016-04-07T04:04:33.35-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25960000 2016-04-07T04:04:33.361-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21400000 QueuedCount 12157 2016-04-07T04:04:33.525-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25970000 2016-04-07T04:04:33.786-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25980000 2016-04-07T04:04:34.013-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 25990000 2016-04-07T04:04:34.033-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21410000 QueuedCount 2157 2016-04-07T04:04:34.199-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26000000 2016-04-07T04:04:34.217-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22779282, 21389641, 0 committed:false 2016-04-07T04:04:34.381-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26010000 2016-04-07T04:04:34.525-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22779282, 21389641, 0 committed:false) SnapCreateDur 13.287µs SnapOpenDur 307.97245ms 2016-04-07T04:04:34.62-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26020000 2016-04-07T04:04:34.89-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26030000 2016-04-07T04:04:35.062-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26040000 2016-04-07T04:04:35.239-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26050000 2016-04-07T04:04:35.431-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26060000 2016-04-07T04:04:35.64-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26070000 2016-04-07T04:04:35.656-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.545µs CommitTime 584.641109ms TotalFlushTime 15m55.209139205s TotalCommitTime 1m18.098181091s 2016-04-07T04:04:35.656-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22793420, 21396710, 185 committed:true 2016-04-07T04:04:35.656-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22793420, 21396710, 185 committed:true) SnapCreateDur 597.648732ms SnapOpenDur 35.2µs 2016-04-07T04:04:35.679-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:35.69-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21420000 QueuedCount 5796 2016-04-07T04:04:35.704-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:35.728-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:35.752-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:35.777-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:35.813-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26080000 2016-04-07T04:04:36.066-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26090000 2016-04-07T04:04:36.265-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26100000 2016-04-07T04:04:36.432-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21430000 QueuedCount 44620 2016-04-07T04:04:36.453-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26110000 2016-04-07T04:04:36.68-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26120000 2016-04-07T04:04:36.949-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26130000 2016-04-07T04:04:37.134-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21440000 QueuedCount 34620 2016-04-07T04:04:37.188-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26140000 2016-04-07T04:04:37.308-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26150000 2016-04-07T04:04:37.455-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26160000 2016-04-07T04:04:37.631-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26170000 2016-04-07T04:04:37.806-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21450000 QueuedCount 24620 2016-04-07T04:04:37.814-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26180000 2016-04-07T04:04:38.023-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26190000 2016-04-07T04:04:38.228-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26200000 2016-04-07T04:04:38.476-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26210000 2016-04-07T04:04:38.818-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26220000 2016-04-07T04:04:39.055-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26230000 2016-04-07T04:04:39.253-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26240000 2016-04-07T04:04:39.455-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26250000 2016-04-07T04:04:39.528-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21460000 QueuedCount 14620 2016-04-07T04:04:39.648-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26260000 2016-04-07T04:04:39.842-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26270000 2016-04-07T04:04:40.059-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26280000 2016-04-07T04:04:40.255-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21470000 QueuedCount 4620 2016-04-07T04:04:40.328-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26290000 2016-04-07T04:04:40.542-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26300000 2016-04-07T04:04:40.55-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22902176, 21451088, 0 committed:false 2016-04-07T04:04:40.743-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26310000 2016-04-07T04:04:40.869-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22902176, 21451088, 0 committed:false) SnapCreateDur 11.928µs SnapOpenDur 319.063105ms 2016-04-07T04:04:40.942-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26320000 2016-04-07T04:04:41.134-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26330000 2016-04-07T04:04:41.263-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21480000 QueuedCount 863 2016-04-07T04:04:41.32-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26340000 2016-04-07T04:04:41.524-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26350000 2016-04-07T04:04:41.723-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26360000 2016-04-07T04:04:41.929-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.54µs CommitTime 634.74914ms TotalFlushTime 16m0.428879225s TotalCommitTime 1m18.732930231s 2016-04-07T04:04:41.929-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 22914462, 21457231, 186 committed:true 2016-04-07T04:04:41.929-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 22914462, 21457231, 186 committed:true) SnapCreateDur 646.171154ms SnapOpenDur 40.824µs 2016-04-07T04:04:41.939-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26370000 2016-04-07T04:04:41.953-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:41.977-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:42.001-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:42.025-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:42.049-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:42.131-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26380000 2016-04-07T04:04:42.361-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26390000 2016-04-07T04:04:42.636-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26400000 2016-04-07T04:04:42.966-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26410000 2016-04-07T04:04:43.185-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26420000 2016-04-07T04:04:43.245-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26430000 2016-04-07T04:04:43.427-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26440000 2016-04-07T04:04:43.623-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26450000 2016-04-07T04:04:43.63-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21490000 QueuedCount 43029 2016-04-07T04:04:43.821-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26460000 2016-04-07T04:04:44.043-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26470000 2016-04-07T04:04:44.281-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26480000 2016-04-07T04:04:44.329-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21500000 QueuedCount 33029 2016-04-07T04:04:44.651-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26490000 2016-04-07T04:04:44.842-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26500000 2016-04-07T04:04:45.056-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26510000 2016-04-07T04:04:45.265-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26520000 2016-04-07T04:04:45.463-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26530000 2016-04-07T04:04:45.686-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26540000 2016-04-07T04:04:45.864-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26550000 2016-04-07T04:04:46.027-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21510000 QueuedCount 23029 2016-04-07T04:04:46.062-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26560000 2016-04-07T04:04:46.272-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26570000 2016-04-07T04:04:46.486-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26580000 2016-04-07T04:04:46.743-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26590000 2016-04-07T04:04:46.785-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21520000 QueuedCount 13029 2016-04-07T04:04:47.065-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26600000 2016-04-07T04:04:47.34-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26610000 2016-04-07T04:04:47.46-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21530000 QueuedCount 3029 2016-04-07T04:04:47.55-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26620000 2016-04-07T04:04:47.651-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23017086, 21508543, 0 committed:false 2016-04-07T04:04:47.77-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26630000 2016-04-07T04:04:47.963-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23017086, 21508543, 0 committed:false) SnapCreateDur 11.692µs SnapOpenDur 312.137095ms 2016-04-07T04:04:47.969-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26640000 2016-04-07T04:04:48.172-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26650000 2016-04-07T04:04:48.407-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26660000 2016-04-07T04:04:48.483-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21540000 QueuedCount 845 2016-04-07T04:04:48.607-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26670000 2016-04-07T04:04:48.791-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26680000 2016-04-07T04:04:49.044-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26690000 2016-04-07T04:04:49.216-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.802µs CommitTime 698.251398ms TotalFlushTime 16m6.614188191s TotalCommitTime 1m19.431181629s 2016-04-07T04:04:49.216-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23032478, 21516239, 187 committed:true 2016-04-07T04:04:49.216-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23032478, 21516239, 187 committed:true) SnapCreateDur 709.540702ms SnapOpenDur 35.147µs 2016-04-07T04:04:49.24-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:49.264-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:49.288-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26700000 2016-04-07T04:04:49.288-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:49.313-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:49.337-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:49.478-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26710000 2016-04-07T04:04:49.686-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26720000 2016-04-07T04:04:49.865-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26730000 2016-04-07T04:04:49.95-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21550000 QueuedCount 43322 2016-04-07T04:04:50.059-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26740000 2016-04-07T04:04:50.259-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26750000 2016-04-07T04:04:50.452-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26760000 2016-04-07T04:04:50.659-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26770000 2016-04-07T04:04:50.722-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21560000 QueuedCount 33322 2016-04-07T04:04:50.85-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26780000 2016-04-07T04:04:51.029-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26790000 2016-04-07T04:04:51.276-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26800000 2016-04-07T04:04:51.417-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21570000 QueuedCount 23322 2016-04-07T04:04:51.605-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26810000 2016-04-07T04:04:51.891-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26820000 2016-04-07T04:04:52.106-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26830000 2016-04-07T04:04:52.335-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26840000 2016-04-07T04:04:52.52-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26850000 2016-04-07T04:04:52.705-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26860000 2016-04-07T04:04:52.876-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26870000 2016-04-07T04:04:53.017-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26880000 2016-04-07T04:04:53.161-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26890000 2016-04-07T04:04:53.256-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21580000 QueuedCount 13322 2016-04-07T04:04:53.375-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26900000 2016-04-07T04:04:53.631-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26910000 2016-04-07T04:04:53.892-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26920000 2016-04-07T04:04:53.998-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21590000 QueuedCount 3322 2016-04-07T04:04:54.124-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26930000 2016-04-07T04:04:54.232-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23135820, 21567910, 0 committed:false 2016-04-07T04:04:54.361-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26940000 2016-04-07T04:04:54.559-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23135820, 21567910, 0 committed:false) SnapCreateDur 14.431µs SnapOpenDur 326.030366ms 2016-04-07T04:04:54.572-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26950000 2016-04-07T04:04:54.786-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26960000 2016-04-07T04:04:54.993-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26970000 2016-04-07T04:04:55.045-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21600000 QueuedCount 2613 2016-04-07T04:04:55.205-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26980000 2016-04-07T04:04:55.415-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 26990000 2016-04-07T04:04:55.633-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27000000 2016-04-07T04:04:55.857-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.619µs CommitTime 602.340959ms TotalFlushTime 16m12.227716292s TotalCommitTime 1m20.033522588s 2016-04-07T04:04:55.857-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23154108, 21577054, 188 committed:true 2016-04-07T04:04:55.857-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23154108, 21577054, 188 committed:true) SnapCreateDur 614.305429ms SnapOpenDur 58.472µs 2016-04-07T04:04:55.883-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:55.898-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27010000 2016-04-07T04:04:55.907-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:55.931-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:55.957-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:55.98-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:04:56.359-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27020000 2016-04-07T04:04:56.456-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21610000 QueuedCount 43535 2016-04-07T04:04:56.529-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27030000 2016-04-07T04:04:56.719-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27040000 2016-04-07T04:04:56.918-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27050000 2016-04-07T04:04:57.11-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27060000 2016-04-07T04:04:57.239-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21620000 QueuedCount 33535 2016-04-07T04:04:57.311-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27070000 2016-04-07T04:04:57.506-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27080000 2016-04-07T04:04:57.69-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27090000 2016-04-07T04:04:57.891-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27100000 2016-04-07T04:04:58.007-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21630000 QueuedCount 23535 2016-04-07T04:04:58.103-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27110000 2016-04-07T04:04:58.317-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27120000 2016-04-07T04:04:58.552-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27130000 2016-04-07T04:04:58.7-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21640000 QueuedCount 13535 2016-04-07T04:04:58.818-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27140000 2016-04-07T04:04:59.016-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27150000 2016-04-07T04:04:59.216-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27160000 2016-04-07T04:04:59.639-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21650000 QueuedCount 3535 2016-04-07T04:04:59.654-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27170000 2016-04-07T04:04:59.72-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27180000 2016-04-07T04:04:59.832-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27190000 2016-04-07T04:04:59.936-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23254224, 21627112, 0 committed:false 2016-04-07T04:05:00.049-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27200000 2016-04-07T04:05:00.261-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23254224, 21627112, 0 committed:false) SnapCreateDur 21.437µs SnapOpenDur 325.57298ms 2016-04-07T04:05:00.33-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27210000 2016-04-07T04:05:00.605-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27220000 2016-04-07T04:05:00.711-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21660000 QueuedCount 2540 2016-04-07T04:05:00.985-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27230000 2016-04-07T04:05:01.258-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27240000 2016-04-07T04:05:01.431-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27250000 2016-04-07T04:05:01.612-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.649µs CommitTime 680.462428ms TotalFlushTime 16m16.876465737s TotalCommitTime 1m20.713985016s 2016-04-07T04:05:01.613-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23271950, 21635975, 189 committed:true 2016-04-07T04:05:01.613-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23271950, 21635975, 189 committed:true) SnapCreateDur 696.626373ms SnapOpenDur 52.509µs 2016-04-07T04:05:01.619-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27260000 2016-04-07T04:05:01.637-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:01.661-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:01.685-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:01.708-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:01.732-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:01.762-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27270000 2016-04-07T04:05:01.972-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27280000 2016-04-07T04:05:02.151-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27290000 2016-04-07T04:05:02.203-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21670000 QueuedCount 42157 2016-04-07T04:05:02.358-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27300000 2016-04-07T04:05:02.548-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27310000 2016-04-07T04:05:02.751-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27320000 2016-04-07T04:05:02.948-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21680000 QueuedCount 32157 2016-04-07T04:05:02.964-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27330000 2016-04-07T04:05:03.18-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27340000 2016-04-07T04:05:03.399-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27350000 2016-04-07T04:05:03.663-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27360000 2016-04-07T04:05:03.678-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21690000 QueuedCount 22157 2016-04-07T04:05:03.898-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27370000 2016-04-07T04:05:04.08-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27380000 2016-04-07T04:05:04.284-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27390000 2016-04-07T04:05:04.399-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21700000 QueuedCount 12157 2016-04-07T04:05:04.531-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27400000 2016-04-07T04:05:04.778-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27410000 2016-04-07T04:05:04.998-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27420000 2016-04-07T04:05:05.096-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21710000 QueuedCount 2157 2016-04-07T04:05:05.218-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27430000 2016-04-07T04:05:05.286-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23369578, 21684789, 0 committed:false 2016-04-07T04:05:05.451-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27440000 2016-04-07T04:05:05.596-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23369578, 21684789, 0 committed:false) SnapCreateDur 12.219µs SnapOpenDur 310.220052ms 2016-04-07T04:05:05.779-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27450000 2016-04-07T04:05:06.069-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27460000 2016-04-07T04:05:06.108-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21720000 QueuedCount 2018 2016-04-07T04:05:06.254-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27470000 2016-04-07T04:05:06.406-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27480000 2016-04-07T04:05:06.608-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27490000 2016-04-07T04:05:06.798-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27500000 2016-04-07T04:05:06.912-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.948µs CommitTime 609.486687ms TotalFlushTime 16m21.161801501s TotalCommitTime 1m21.323471703s 2016-04-07T04:05:06.912-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23388998, 21694499, 190 committed:true 2016-04-07T04:05:06.912-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23388998, 21694499, 190 committed:true) SnapCreateDur 620.936356ms SnapOpenDur 37.492µs 2016-04-07T04:05:06.938-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:06.962-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:06.987-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:07.011-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:07.012-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27510000 2016-04-07T04:05:07.034-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:07.224-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27520000 2016-04-07T04:05:07.442-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27530000 2016-04-07T04:05:07.493-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21730000 QueuedCount 43946 2016-04-07T04:05:07.654-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27540000 2016-04-07T04:05:07.902-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27550000 2016-04-07T04:05:08.11-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27560000 2016-04-07T04:05:08.189-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21740000 QueuedCount 33946 2016-04-07T04:05:08.32-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27570000 2016-04-07T04:05:08.599-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27580000 2016-04-07T04:05:08.876-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21750000 QueuedCount 23946 2016-04-07T04:05:08.882-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27590000 2016-04-07T04:05:09.142-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27600000 2016-04-07T04:05:09.407-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27610000 2016-04-07T04:05:09.541-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21760000 QueuedCount 13946 2016-04-07T04:05:09.606-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27620000 2016-04-07T04:05:09.818-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27630000 2016-04-07T04:05:10.026-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27640000 2016-04-07T04:05:10.225-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27650000 2016-04-07T04:05:10.283-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21770000 QueuedCount 3946 2016-04-07T04:05:10.418-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27660000 2016-04-07T04:05:10.492-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23491274, 21745637, 0 committed:false 2016-04-07T04:05:10.683-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27670000 2016-04-07T04:05:10.801-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23491274, 21745637, 0 committed:false) SnapCreateDur 11.434µs SnapOpenDur 308.651058ms 2016-04-07T04:05:11.045-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27680000 2016-04-07T04:05:11.204-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27690000 2016-04-07T04:05:11.374-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27700000 2016-04-07T04:05:11.56-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27710000 2016-04-07T04:05:11.755-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27720000 2016-04-07T04:05:11.963-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27730000 2016-04-07T04:05:12.181-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27740000 2016-04-07T04:05:12.318-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21780000 QueuedCount 2336 2016-04-07T04:05:12.387-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27750000 2016-04-07T04:05:12.602-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27760000 2016-04-07T04:05:12.818-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27770000 2016-04-07T04:05:12.947-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":5993923,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3076842580,"bucket-1:myindex:avg_ts_items_count":37517,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6892177953,"bucket-1:myindex:delete_bytes":74531642,"bucket-1:myindex:disk_size":26742325248,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":74,"bucket-1:myindex:get_bytes":75740286,"bucket-1:myindex:insert_bytes":2281995765,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":190,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":22014374,"bucket-1:myindex:num_docs_pending":13621,"bucket-1:myindex:num_docs_queued":6061031,"bucket-1:myindex:num_flush_queued":21782336,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":229,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":3889048385,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"190 81323471703 2060050267172960367","bucket-1:myindex:timings/storage_del":"1753889 41673189608 2436026504925279320","bucket-1:myindex:timings/storage_get":"21782336 145422293744 640194295003972244","bucket-1:myindex:timings/storage_handle_open":"190 4583993 115760885525","bucket-1:myindex:timings/storage_info":"842 7462446 184422449254","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"194 156443578 3669451447944192","bucket-1:myindex:timings/storage_meta_set":"191 404278231 107846415590587791","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"43507778 785015565384 8465311383470702922","bucket-1:myindex:timings/storage_snapshot_close":"38 1129978573 36822180383278561","bucket-1:myindex:timings/storage_snapshot_create":"39 14547273032 -3924649490208587892","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":27776259,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":660,"memory_quota":31457280000,"memory_used":10593976320,"needs_restart":false,"num_connections":0,"timings/stats_response":"3954 557471992 96105086422304"} 2016-04-07T04:05:13.03-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27780000 2016-04-07T04:05:13.098-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.888µs CommitTime 652.341446ms TotalFlushTime 16m26.292619324s TotalCommitTime 1m21.975813149s 2016-04-07T04:05:13.098-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23507778, 21753889, 191 committed:true 2016-04-07T04:05:13.098-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23507778, 21753889, 191 committed:true) SnapCreateDur 663.693922ms SnapOpenDur 43.693µs 2016-04-07T04:05:13.123-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:13.148-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:13.173-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:13.197-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:13.222-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:13.25-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27790000 2016-04-07T04:05:13.487-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27800000 2016-04-07T04:05:13.812-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27810000 2016-04-07T04:05:14.096-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27820000 2016-04-07T04:05:14.159-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21790000 QueuedCount 49397 2016-04-07T04:05:14.366-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27830000 2016-04-07T04:05:14.506-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27840000 2016-04-07T04:05:14.717-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27850000 2016-04-07T04:05:14.871-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21800000 QueuedCount 39397 2016-04-07T04:05:14.929-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27860000 2016-04-07T04:05:15.136-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27870000 2016-04-07T04:05:15.334-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27880000 2016-04-07T04:05:15.544-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27890000 2016-04-07T04:05:15.575-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21810000 QueuedCount 29397 2016-04-07T04:05:15.792-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27900000 2016-04-07T04:05:16.199-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27910000 2016-04-07T04:05:16.243-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21820000 QueuedCount 19397 2016-04-07T04:05:16.473-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27920000 2016-04-07T04:05:16.696-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27930000 2016-04-07T04:05:16.871-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27940000 2016-04-07T04:05:17.072-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27950000 2016-04-07T04:05:17.271-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27960000 2016-04-07T04:05:17.491-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27970000 2016-04-07T04:05:17.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27980000 2016-04-07T04:05:17.86-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 27990000 2016-04-07T04:05:18.072-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28000000 2016-04-07T04:05:18.325-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28010000 2016-04-07T04:05:18.602-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28020000 2016-04-07T04:05:18.804-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28030000 2016-04-07T04:05:18.982-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21830000 QueuedCount 9397 2016-04-07T04:05:19.013-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28040000 2016-04-07T04:05:19.24-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28050000 2016-04-07T04:05:19.51-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28060000 2016-04-07T04:05:19.65-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23620030, 21810015, 0 committed:false 2016-04-07T04:05:19.717-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28070000 2016-04-07T04:05:19.919-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28080000 2016-04-07T04:05:19.982-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23620030, 21810015, 0 committed:false) SnapCreateDur 14.608µs SnapOpenDur 331.592213ms 2016-04-07T04:05:20.085-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21840000 QueuedCount 6658 2016-04-07T04:05:20.136-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28090000 2016-04-07T04:05:20.357-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28100000 2016-04-07T04:05:20.582-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28110000 2016-04-07T04:05:20.81-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28120000 2016-04-07T04:05:21.02-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28130000 2016-04-07T04:05:21.186-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.875µs CommitTime 678.675536ms TotalFlushTime 16m33.265657025s TotalCommitTime 1m22.654488685s 2016-04-07T04:05:21.186-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23634352, 21817176, 192 committed:true 2016-04-07T04:05:21.186-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23634352, 21817176, 192 committed:true) SnapCreateDur 690.304187ms SnapOpenDur 42.99µs 2016-04-07T04:05:21.209-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:21.234-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28140000 2016-04-07T04:05:21.234-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:21.259-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:21.285-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:21.309-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:21.441-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21850000 QueuedCount 50000 2016-04-07T04:05:21.625-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28150000 2016-04-07T04:05:21.923-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28160000 2016-04-07T04:05:22.086-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28170000 2016-04-07T04:05:22.182-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21860000 QueuedCount 45966 2016-04-07T04:05:22.275-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28180000 2016-04-07T04:05:22.5-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28190000 2016-04-07T04:05:22.706-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28200000 2016-04-07T04:05:22.873-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21870000 QueuedCount 35966 2016-04-07T04:05:22.901-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28210000 2016-04-07T04:05:23.112-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28220000 2016-04-07T04:05:23.368-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28230000 2016-04-07T04:05:23.566-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21880000 QueuedCount 25966 2016-04-07T04:05:23.631-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28240000 2016-04-07T04:05:23.816-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28250000 2016-04-07T04:05:24.028-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28260000 2016-04-07T04:05:24.231-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21890000 QueuedCount 15966 2016-04-07T04:05:24.236-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28270000 2016-04-07T04:05:24.428-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28280000 2016-04-07T04:05:24.649-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28290000 2016-04-07T04:05:24.941-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28300000 2016-04-07T04:05:25.019-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21900000 QueuedCount 5966 2016-04-07T04:05:25.191-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28310000 2016-04-07T04:05:25.367-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28320000 2016-04-07T04:05:25.412-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23750970, 21875485, 0 committed:false 2016-04-07T04:05:25.556-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28330000 2016-04-07T04:05:25.739-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28340000 2016-04-07T04:05:25.759-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23750970, 21875485, 0 committed:false) SnapCreateDur 14.494µs SnapOpenDur 347.570849ms 2016-04-07T04:05:25.932-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28350000 2016-04-07T04:05:26.079-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21910000 QueuedCount 3923 2016-04-07T04:05:26.16-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28360000 2016-04-07T04:05:26.399-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28370000 2016-04-07T04:05:26.619-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28380000 2016-04-07T04:05:26.878-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28390000 2016-04-07T04:05:27.203-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.933µs CommitTime 826.604216ms TotalFlushTime 16m37.996887032s TotalCommitTime 1m23.481092901s 2016-04-07T04:05:27.203-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23766664, 21883332, 193 committed:true 2016-04-07T04:05:27.203-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23766664, 21883332, 193 committed:true) SnapCreateDur 839.128507ms SnapOpenDur 30.824µs 2016-04-07T04:05:27.227-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:27.25-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:27.266-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28400000 2016-04-07T04:05:27.275-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:27.301-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:27.326-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:27.483-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28410000 2016-04-07T04:05:27.668-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28420000 2016-04-07T04:05:27.878-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28430000 2016-04-07T04:05:28.104-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28440000 2016-04-07T04:05:28.31-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28450000 2016-04-07T04:05:28.481-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28460000 2016-04-07T04:05:28.692-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28470000 2016-04-07T04:05:28.758-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21920000 QueuedCount 49783 2016-04-07T04:05:28.935-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28480000 2016-04-07T04:05:29.15-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28490000 2016-04-07T04:05:29.336-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28500000 2016-04-07T04:05:29.561-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28510000 2016-04-07T04:05:29.764-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28520000 2016-04-07T04:05:29.968-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28530000 2016-04-07T04:05:30.185-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28540000 2016-04-07T04:05:30.441-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28550000 2016-04-07T04:05:30.523-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21930000 QueuedCount 39783 2016-04-07T04:05:30.728-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28560000 2016-04-07T04:05:30.958-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28570000 2016-04-07T04:05:31.154-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28580000 2016-04-07T04:05:31.229-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21940000 QueuedCount 29783 2016-04-07T04:05:31.357-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28590000 2016-04-07T04:05:31.564-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28600000 2016-04-07T04:05:31.768-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28610000 2016-04-07T04:05:31.918-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21950000 QueuedCount 19783 2016-04-07T04:05:31.999-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28620000 2016-04-07T04:05:32.231-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28630000 2016-04-07T04:05:32.443-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28640000 2016-04-07T04:05:32.584-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21960000 QueuedCount 9783 2016-04-07T04:05:32.904-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28650000 2016-04-07T04:05:33.197-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28660000 2016-04-07T04:05:33.264-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23876628, 21938314, 0 committed:false 2016-04-07T04:05:33.37-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28670000 2016-04-07T04:05:33.54-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28680000 2016-04-07T04:05:33.6-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23876628, 21938314, 0 committed:false) SnapCreateDur 14.647µs SnapOpenDur 336.110947ms 2016-04-07T04:05:33.7-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21970000 QueuedCount 9187 2016-04-07T04:05:33.731-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28690000 2016-04-07T04:05:33.937-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28700000 2016-04-07T04:05:34.132-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28710000 2016-04-07T04:05:34.337-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28720000 2016-04-07T04:05:34.535-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28730000 2016-04-07T04:05:34.727-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28740000 2016-04-07T04:05:35.216-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.426µs CommitTime 897.295088ms TotalFlushTime 16m44.676266212s TotalCommitTime 1m24.378387989s 2016-04-07T04:05:35.216-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23895138, 21947569, 194 committed:true 2016-04-07T04:05:35.216-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23895138, 21947569, 194 committed:true) SnapCreateDur 909.264497ms SnapOpenDur 60.265µs 2016-04-07T04:05:35.241-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:35.25-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28750000 2016-04-07T04:05:35.253-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21980000 QueuedCount 7190 2016-04-07T04:05:35.264-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:35.288-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:35.295-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28760000 2016-04-07T04:05:35.314-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:35.339-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:35.371-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28770000 2016-04-07T04:05:35.594-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28780000 2016-04-07T04:05:35.817-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28790000 2016-04-07T04:05:36.022-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28800000 2016-04-07T04:05:36.055-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 21990000 QueuedCount 40738 2016-04-07T04:05:36.28-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28810000 2016-04-07T04:05:36.544-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28820000 2016-04-07T04:05:36.748-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28830000 2016-04-07T04:05:36.799-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22000000 QueuedCount 30738 2016-04-07T04:05:36.962-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28840000 2016-04-07T04:05:37.153-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28850000 2016-04-07T04:05:37.342-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28860000 2016-04-07T04:05:37.572-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28870000 2016-04-07T04:05:37.58-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22010000 QueuedCount 20738 2016-04-07T04:05:37.75-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28880000 2016-04-07T04:05:37.953-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28890000 2016-04-07T04:05:38.265-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22020000 QueuedCount 10738 2016-04-07T04:05:38.407-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28900000 2016-04-07T04:05:38.692-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28910000 2016-04-07T04:05:38.859-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28920000 2016-04-07T04:05:39.022-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28930000 2016-04-07T04:05:39.216-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28940000 2016-04-07T04:05:39.432-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28950000 2016-04-07T04:05:39.629-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28960000 2016-04-07T04:05:39.811-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28970000 2016-04-07T04:05:40.022-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28980000 2016-04-07T04:05:40.025-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22030000 QueuedCount 738 2016-04-07T04:05:40.038-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 23996666, 21998333, 0 committed:false 2016-04-07T04:05:40.213-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 28990000 2016-04-07T04:05:40.356-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 23996666, 21998333, 0 committed:false) SnapCreateDur 11.909µs SnapOpenDur 318.168554ms 2016-04-07T04:05:40.437-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29000000 2016-04-07T04:05:40.653-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29010000 2016-04-07T04:05:40.857-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29020000 2016-04-07T04:05:41.037-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22040000 QueuedCount 2983 2016-04-07T04:05:41.085-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29030000 2016-04-07T04:05:41.326-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29040000 2016-04-07T04:05:41.57-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29050000 2016-04-07T04:05:41.778-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29060000 2016-04-07T04:05:41.914-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.747µs CommitTime 670.783744ms TotalFlushTime 16m50.298728298s TotalCommitTime 1m25.049171733s 2016-04-07T04:05:41.914-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24020766, 22010383, 195 committed:true 2016-04-07T04:05:41.914-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24020766, 22010383, 195 committed:true) SnapCreateDur 682.612211ms SnapOpenDur 35.091µs 2016-04-07T04:05:41.939-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:41.963-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:41.986-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:42.01-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:42.033-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:42.036-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29070000 2016-04-07T04:05:42.325-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29080000 2016-04-07T04:05:42.484-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22050000 QueuedCount 40480 2016-04-07T04:05:42.625-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29090000 2016-04-07T04:05:42.821-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29100000 2016-04-07T04:05:43.036-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29110000 2016-04-07T04:05:43.197-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22060000 QueuedCount 30480 2016-04-07T04:05:43.245-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29120000 2016-04-07T04:05:43.492-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29130000 2016-04-07T04:05:43.748-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29140000 2016-04-07T04:05:43.895-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22070000 QueuedCount 20480 2016-04-07T04:05:43.961-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29150000 2016-04-07T04:05:44.211-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29160000 2016-04-07T04:05:44.575-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22080000 QueuedCount 10480 2016-04-07T04:05:44.629-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29170000 2016-04-07T04:05:44.856-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29180000 2016-04-07T04:05:45.013-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29190000 2016-04-07T04:05:45.196-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29200000 2016-04-07T04:05:45.252-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22090000 QueuedCount 480 2016-04-07T04:05:45.267-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24114286, 22057143, 0 committed:false 2016-04-07T04:05:45.389-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29210000 2016-04-07T04:05:45.555-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29220000 2016-04-07T04:05:45.604-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24114286, 22057143, 0 committed:false) SnapCreateDur 12.976µs SnapOpenDur 336.554011ms 2016-04-07T04:05:45.746-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29230000 2016-04-07T04:05:45.942-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29240000 2016-04-07T04:05:46.161-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29250000 2016-04-07T04:05:46.304-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22100000 QueuedCount 2612 2016-04-07T04:05:46.381-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29260000 2016-04-07T04:05:46.606-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29270000 2016-04-07T04:05:46.84-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29280000 2016-04-07T04:05:47.052-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29290000 2016-04-07T04:05:47.195-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.932µs CommitTime 673.189183ms TotalFlushTime 16m54.475423699s TotalCommitTime 1m25.722360916s 2016-04-07T04:05:47.195-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24138150, 22069075, 196 committed:true 2016-04-07T04:05:47.195-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24138150, 22069075, 196 committed:true) SnapCreateDur 686.074069ms SnapOpenDur 35.885µs 2016-04-07T04:05:47.22-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:47.244-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:47.268-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:47.273-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29300000 2016-04-07T04:05:47.292-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:47.316-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:47.505-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29310000 2016-04-07T04:05:47.733-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29320000 2016-04-07T04:05:47.973-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29330000 2016-04-07T04:05:48.196-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29340000 2016-04-07T04:05:48.226-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22110000 QueuedCount 48080 2016-04-07T04:05:48.481-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29350000 2016-04-07T04:05:48.758-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29360000 2016-04-07T04:05:48.95-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22120000 QueuedCount 38080 2016-04-07T04:05:49.003-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29370000 2016-04-07T04:05:49.194-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29380000 2016-04-07T04:05:49.399-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29390000 2016-04-07T04:05:49.619-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29400000 2016-04-07T04:05:49.857-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29410000 2016-04-07T04:05:50.064-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29420000 2016-04-07T04:05:50.342-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29430000 2016-04-07T04:05:50.718-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22130000 QueuedCount 28080 2016-04-07T04:05:50.754-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29440000 2016-04-07T04:05:51.023-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29450000 2016-04-07T04:05:51.194-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29460000 2016-04-07T04:05:51.406-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29470000 2016-04-07T04:05:51.407-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22140000 QueuedCount 18080 2016-04-07T04:05:51.593-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29480000 2016-04-07T04:05:51.777-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29490000 2016-04-07T04:05:51.968-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29500000 2016-04-07T04:05:52.097-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22150000 QueuedCount 8080 2016-04-07T04:05:52.178-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29510000 2016-04-07T04:05:52.376-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29520000 2016-04-07T04:05:52.592-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29530000 2016-04-07T04:05:52.668-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24247358, 22123679, 0 committed:false 2016-04-07T04:05:52.801-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29540000 2016-04-07T04:05:53.007-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24247358, 22123679, 0 committed:false) SnapCreateDur 10.052µs SnapOpenDur 338.813578ms 2016-04-07T04:05:53.031-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29550000 2016-04-07T04:05:53.132-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22160000 QueuedCount 3264 2016-04-07T04:05:53.262-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29560000 2016-04-07T04:05:53.488-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29570000 2016-04-07T04:05:53.722-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29580000 2016-04-07T04:05:53.955-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29590000 2016-04-07T04:05:54.167-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29600000 2016-04-07T04:05:54.389-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29610000 2016-04-07T04:05:54.742-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29620000 2016-04-07T04:05:55.084-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.503µs CommitTime 623.295022ms TotalFlushTime 17m1.309505862s TotalCommitTime 1m26.345655938s 2016-04-07T04:05:55.086-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24257556, 22128778, 197 committed:true 2016-04-07T04:05:55.086-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24257556, 22128778, 197 committed:true) SnapCreateDur 636.122136ms SnapOpenDur 39.363µs 2016-04-07T04:05:55.102-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29630000 2016-04-07T04:05:55.111-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:55.136-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:55.161-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:55.186-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:55.21-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:05:55.276-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29640000 2016-04-07T04:05:55.484-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29650000 2016-04-07T04:05:55.579-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22170000 QueuedCount 50000 2016-04-07T04:05:55.692-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29660000 2016-04-07T04:05:55.887-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29670000 2016-04-07T04:05:56.075-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29680000 2016-04-07T04:05:56.302-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29690000 2016-04-07T04:05:56.392-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22180000 QueuedCount 42560 2016-04-07T04:05:56.518-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29700000 2016-04-07T04:05:56.838-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29710000 2016-04-07T04:05:57.086-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22190000 QueuedCount 32560 2016-04-07T04:05:57.149-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29720000 2016-04-07T04:05:57.298-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29730000 2016-04-07T04:05:57.47-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29740000 2016-04-07T04:05:57.649-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29750000 2016-04-07T04:05:57.844-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29760000 2016-04-07T04:05:58.037-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29770000 2016-04-07T04:05:58.236-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29780000 2016-04-07T04:05:58.467-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29790000 2016-04-07T04:05:58.684-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29800000 2016-04-07T04:05:58.88-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29810000 2016-04-07T04:05:59.104-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29820000 2016-04-07T04:05:59.322-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29830000 2016-04-07T04:05:59.546-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29840000 2016-04-07T04:05:59.777-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29850000 2016-04-07T04:06:00.042-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29860000 2016-04-07T04:06:00.222-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22200000 QueuedCount 22560 2016-04-07T04:06:00.305-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29870000 2016-04-07T04:06:00.597-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29880000 2016-04-07T04:06:00.8-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29890000 2016-04-07T04:06:01.005-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29900000 2016-04-07T04:06:01.254-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29910000 2016-04-07T04:06:01.536-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29920000 2016-04-07T04:06:01.627-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22210000 QueuedCount 12560 2016-04-07T04:06:01.781-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29930000 2016-04-07T04:06:01.959-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29940000 2016-04-07T04:06:02.14-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29950000 2016-04-07T04:06:02.313-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22220000 QueuedCount 2560 2016-04-07T04:06:02.334-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29960000 2016-04-07T04:06:02.501-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24374238, 22187119, 0 committed:false 2016-04-07T04:06:02.537-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29970000 2016-04-07T04:06:02.746-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29980000 2016-04-07T04:06:02.852-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24374238, 22187119, 0 committed:false) SnapCreateDur 11.237µs SnapOpenDur 350.309368ms 2016-04-07T04:06:02.961-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 29990000 2016-04-07T04:06:03.338-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30000000 2016-04-07T04:06:03.353-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22230000 QueuedCount 2112 2016-04-07T04:06:03.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30010000 2016-04-07T04:06:03.816-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30020000 2016-04-07T04:06:04.006-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30030000 2016-04-07T04:06:04.181-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30040000 2016-04-07T04:06:04.348-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.205µs CommitTime 799.7711ms TotalFlushTime 17m9.315303996s TotalCommitTime 1m27.145427038s 2016-04-07T04:06:04.348-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24393046, 22196523, 198 committed:true 2016-04-07T04:06:04.348-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24393046, 22196523, 198 committed:true) SnapCreateDur 811.60712ms SnapOpenDur 34.433µs 2016-04-07T04:06:04.372-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:04.375-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30050000 2016-04-07T04:06:04.397-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:04.421-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:04.446-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:04.47-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:04.555-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30060000 2016-04-07T04:06:04.764-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30070000 2016-04-07T04:06:04.949-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22240000 QueuedCount 42341 2016-04-07T04:06:04.98-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30080000 2016-04-07T04:06:05.195-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30090000 2016-04-07T04:06:05.408-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30100000 2016-04-07T04:06:05.627-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30110000 2016-04-07T04:06:05.707-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22250000 QueuedCount 32341 2016-04-07T04:06:05.861-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30120000 2016-04-07T04:06:06.163-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30130000 2016-04-07T04:06:06.417-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30140000 2016-04-07T04:06:06.45-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22260000 QueuedCount 22341 2016-04-07T04:06:06.63-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30150000 2016-04-07T04:06:06.836-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30160000 2016-04-07T04:06:07.067-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30170000 2016-04-07T04:06:07.281-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30180000 2016-04-07T04:06:07.483-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30190000 2016-04-07T04:06:07.696-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30200000 2016-04-07T04:06:07.993-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30210000 2016-04-07T04:06:08.282-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30220000 2016-04-07T04:06:08.282-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22270000 QueuedCount 12341 2016-04-07T04:06:08.482-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30230000 2016-04-07T04:06:08.653-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30240000 2016-04-07T04:06:08.841-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30250000 2016-04-07T04:06:08.98-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22280000 QueuedCount 2341 2016-04-07T04:06:09.027-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30260000 2016-04-07T04:06:09.167-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24491908, 22245954, 0 committed:false 2016-04-07T04:06:09.233-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30270000 2016-04-07T04:06:09.452-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30280000 2016-04-07T04:06:09.498-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24491908, 22245954, 0 committed:false) SnapCreateDur 11.869µs SnapOpenDur 330.865588ms 2016-04-07T04:06:09.783-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30290000 2016-04-07T04:06:10.003-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22290000 QueuedCount 2095 2016-04-07T04:06:10.144-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30300000 2016-04-07T04:06:10.37-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30310000 2016-04-07T04:06:10.543-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30320000 2016-04-07T04:06:10.737-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30330000 2016-04-07T04:06:10.806-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.246µs CommitTime 612.385201ms TotalFlushTime 17m14.739842795s TotalCommitTime 1m27.757812239s 2016-04-07T04:06:10.806-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24511140, 22255570, 199 committed:true 2016-04-07T04:06:10.806-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24511140, 22255570, 199 committed:true) SnapCreateDur 624.225555ms SnapOpenDur 40.684µs 2016-04-07T04:06:10.832-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:10.856-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:10.88-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:10.904-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:10.916-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30340000 2016-04-07T04:06:10.927-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:11.103-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30350000 2016-04-07T04:06:11.278-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30360000 2016-04-07T04:06:11.38-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22300000 QueuedCount 40049 2016-04-07T04:06:11.495-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30370000 2016-04-07T04:06:11.701-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30380000 2016-04-07T04:06:11.956-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30390000 2016-04-07T04:06:12.236-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30400000 2016-04-07T04:06:12.449-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30410000 2016-04-07T04:06:12.644-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30420000 2016-04-07T04:06:12.86-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30430000 2016-04-07T04:06:12.947-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8094268,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2807897421,"bucket-1:myindex:avg_ts_items_count":23741,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":6987967933,"bucket-1:myindex:delete_bytes":96870572,"bucket-1:myindex:disk_size":30341521408,"bucket-1:myindex:flush_queue_size":32639,"bucket-1:myindex:frag_percent":76,"bucket-1:myindex:get_bytes":98439470,"bucket-1:myindex:insert_bytes":2336544377,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":199,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":22547858,"bucket-1:myindex:num_docs_pending":15542,"bucket-1:myindex:num_docs_queued":8323170,"bucket-1:myindex:num_flush_queued":22340049,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":246,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1308880360,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"199 87757812239 6740893755936986889","bucket-1:myindex:timings/storage_del":"2270651 56531932358 3156820449825361692","bucket-1:myindex:timings/storage_get":"22307411 150333985509 640242952242250693","bucket-1:myindex:timings/storage_handle_open":"199 4789475 120677489349","bucket-1:myindex:timings/storage_info":"891 7786131 189150069467","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"202 159436620 3670576724893932","bucket-1:myindex:timings/storage_meta_set":"199 407734852 107847910361881310","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"44541301 815435479091 -8459334936156910519","bucket-1:myindex:timings/storage_snapshot_close":"47 1487843905 51081023718505615","bucket-1:myindex:timings/storage_snapshot_create":"47 17236813623 -3019735566875943297","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":30434317,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":897,"memory_quota":31457280000,"memory_used":12168118272,"needs_restart":false,"num_connections":0,"timings/stats_response":"4038 572367678 98839459664082"} 2016-04-07T04:06:13.061-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30440000 2016-04-07T04:06:13.147-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22310000 QueuedCount 30049 2016-04-07T04:06:13.27-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30450000 2016-04-07T04:06:13.489-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30460000 2016-04-07T04:06:13.704-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30470000 2016-04-07T04:06:13.854-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22320000 QueuedCount 20049 2016-04-07T04:06:13.921-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30480000 2016-04-07T04:06:14.142-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30490000 2016-04-07T04:06:14.345-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30500000 2016-04-07T04:06:14.586-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30510000 2016-04-07T04:06:14.856-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30520000 2016-04-07T04:06:15.135-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30530000 2016-04-07T04:06:15.331-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30540000 2016-04-07T04:06:15.541-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30550000 2016-04-07T04:06:15.71-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22330000 QueuedCount 10049 2016-04-07T04:06:15.739-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30560000 2016-04-07T04:06:15.961-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30570000 2016-04-07T04:06:16.183-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30580000 2016-04-07T04:06:16.404-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22340000 QueuedCount 49 2016-04-07T04:06:16.41-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24605502, 22302751, 0 committed:false 2016-04-07T04:06:16.473-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30590000 2016-04-07T04:06:16.732-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24605502, 22302751, 0 committed:false) SnapCreateDur 9.574µs SnapOpenDur 321.916327ms 2016-04-07T04:06:16.84-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30600000 2016-04-07T04:06:17.057-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30610000 2016-04-07T04:06:17.201-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30620000 2016-04-07T04:06:17.346-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30630000 2016-04-07T04:06:17.901-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30640000 2016-04-07T04:06:17.966-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30650000 2016-04-07T04:06:18.03-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30660000 2016-04-07T04:06:18.259-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30670000 2016-04-07T04:06:18.425-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30680000 2016-04-07T04:06:18.518-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.558µs CommitTime 663.516918ms TotalFlushTime 17m20.899773484s TotalCommitTime 1m28.421329157s 2016-04-07T04:06:18.518-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24623052, 22311526, 200 committed:true 2016-04-07T04:06:18.518-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24623052, 22311526, 200 committed:true) SnapCreateDur 1.150354544s SnapOpenDur 51.058µs 2016-04-07T04:06:18.544-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:18.559-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22350000 QueuedCount 11029 2016-04-07T04:06:18.57-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:18.582-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30690000 2016-04-07T04:06:18.595-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:18.619-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:18.644-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:18.732-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30700000 2016-04-07T04:06:18.911-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30710000 2016-04-07T04:06:19.135-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30720000 2016-04-07T04:06:19.362-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30730000 2016-04-07T04:06:19.599-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30740000 2016-04-07T04:06:19.818-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30750000 2016-04-07T04:06:20.052-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30760000 2016-04-07T04:06:20.256-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30770000 2016-04-07T04:06:20.4-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22360000 QueuedCount 47051 2016-04-07T04:06:20.468-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30780000 2016-04-07T04:06:20.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30790000 2016-04-07T04:06:20.898-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30800000 2016-04-07T04:06:21.124-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30810000 2016-04-07T04:06:21.194-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22370000 QueuedCount 37051 2016-04-07T04:06:21.35-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30820000 2016-04-07T04:06:21.596-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30830000 2016-04-07T04:06:21.893-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22380000 QueuedCount 27051 2016-04-07T04:06:21.894-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30840000 2016-04-07T04:06:22.172-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30850000 2016-04-07T04:06:22.469-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30860000 2016-04-07T04:06:22.57-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22390000 QueuedCount 17051 2016-04-07T04:06:22.617-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30870000 2016-04-07T04:06:22.78-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30880000 2016-04-07T04:06:22.959-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30890000 2016-04-07T04:06:23.157-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30900000 2016-04-07T04:06:23.259-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22400000 QueuedCount 7051 2016-04-07T04:06:23.442-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30910000 2016-04-07T04:06:23.74-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24737388, 22368694, 0 committed:false 2016-04-07T04:06:23.931-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30920000 2016-04-07T04:06:24.092-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24737388, 22368694, 0 committed:false) SnapCreateDur 16.966µs SnapOpenDur 352.606378ms 2016-04-07T04:06:24.192-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30930000 2016-04-07T04:06:24.308-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22410000 QueuedCount 5174 2016-04-07T04:06:24.385-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30940000 2016-04-07T04:06:24.582-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30950000 2016-04-07T04:06:24.785-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30960000 2016-04-07T04:06:25.016-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30970000 2016-04-07T04:06:25.196-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30980000 2016-04-07T04:06:25.353-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.578µs CommitTime 664.641966ms TotalFlushTime 17m26.618745392s TotalCommitTime 1m29.085971123s 2016-04-07T04:06:25.353-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24753370, 22376685, 201 committed:true 2016-04-07T04:06:25.353-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24753370, 22376685, 201 committed:true) SnapCreateDur 675.924756ms SnapOpenDur 34.59µs 2016-04-07T04:06:25.377-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:25.399-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:25.411-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 30990000 2016-04-07T04:06:25.423-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:25.448-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:25.472-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:25.578-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31000000 2016-04-07T04:06:25.724-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22420000 QueuedCount 45662 2016-04-07T04:06:25.77-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31010000 2016-04-07T04:06:25.963-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31020000 2016-04-07T04:06:26.167-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31030000 2016-04-07T04:06:26.352-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31040000 2016-04-07T04:06:26.446-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22430000 QueuedCount 35662 2016-04-07T04:06:26.572-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31050000 2016-04-07T04:06:26.766-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31060000 2016-04-07T04:06:26.996-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31070000 2016-04-07T04:06:27.167-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22440000 QueuedCount 25662 2016-04-07T04:06:27.206-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31080000 2016-04-07T04:06:27.411-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31090000 2016-04-07T04:06:27.627-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31100000 2016-04-07T04:06:27.851-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22450000 QueuedCount 15662 2016-04-07T04:06:27.856-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31110000 2016-04-07T04:06:28.072-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31120000 2016-04-07T04:06:28.277-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31130000 2016-04-07T04:06:28.489-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31140000 2016-04-07T04:06:28.616-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22460000 QueuedCount 5662 2016-04-07T04:06:28.711-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31150000 2016-04-07T04:06:28.974-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31160000 2016-04-07T04:06:29.007-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24852660, 22426330, 0 committed:false 2016-04-07T04:06:29.34-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24852660, 22426330, 0 committed:false) SnapCreateDur 13.011µs SnapOpenDur 333.074154ms 2016-04-07T04:06:29.663-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22470000 QueuedCount 5492 2016-04-07T04:06:31.778-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.739µs CommitTime 640.762731ms TotalFlushTime 17m31.978488004s TotalCommitTime 1m29.726733854s 2016-04-07T04:06:31.778-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24871960, 22435980, 202 committed:true 2016-04-07T04:06:31.778-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24871960, 22435980, 202 committed:true) SnapCreateDur 652.591118ms SnapOpenDur 36.706µs 2016-04-07T04:06:31.802-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:31.811-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31170000 2016-04-07T04:06:31.827-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:31.851-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:31.855-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31180000 2016-04-07T04:06:31.874-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:31.897-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:31.898-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31190000 2016-04-07T04:06:31.941-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31200000 2016-04-07T04:06:31.991-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31210000 2016-04-07T04:06:32.043-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31220000 2016-04-07T04:06:32.103-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31230000 2016-04-07T04:06:32.139-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22480000 QueuedCount 41935 2016-04-07T04:06:32.164-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31240000 2016-04-07T04:06:32.896-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22490000 QueuedCount 31935 2016-04-07T04:06:33.619-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22500000 QueuedCount 21935 2016-04-07T04:06:34.323-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22510000 QueuedCount 11935 2016-04-07T04:06:35.014-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22520000 QueuedCount 1935 2016-04-07T04:06:35.201-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24963402, 22481701, 0 committed:false 2016-04-07T04:06:35.538-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24963402, 22481701, 0 committed:false) SnapCreateDur 17.722µs SnapOpenDur 337.627785ms 2016-04-07T04:06:35.582-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31250000 2016-04-07T04:06:35.632-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31260000 2016-04-07T04:06:37.094-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22530000 QueuedCount 437 2016-04-07T04:06:37.748-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.708µs CommitTime 634.320175ms TotalFlushTime 17m36.890700136s TotalCommitTime 1m30.361054029s 2016-04-07T04:06:37.748-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 24980142, 22490071, 203 committed:true 2016-04-07T04:06:37.748-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 24980142, 22490071, 203 committed:true) SnapCreateDur 645.59741ms SnapOpenDur 36.848µs 2016-04-07T04:06:37.773-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:37.798-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:37.823-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:37.843-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31270000 2016-04-07T04:06:37.847-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:37.871-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:37.884-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31280000 2016-04-07T04:06:37.924-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31290000 2016-04-07T04:06:37.971-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31300000 2016-04-07T04:06:38.017-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31310000 2016-04-07T04:06:38.076-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31320000 2016-04-07T04:06:38.473-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22540000 QueuedCount 49211 2016-04-07T04:06:39.285-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22550000 QueuedCount 39211 2016-04-07T04:06:39.998-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22560000 QueuedCount 29211 2016-04-07T04:06:40.67-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22570000 QueuedCount 19211 2016-04-07T04:06:41.345-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22580000 QueuedCount 9211 2016-04-07T04:06:41.981-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25095816, 22547908, 0 committed:false 2016-04-07T04:06:42.341-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25095816, 22547908, 0 committed:false) SnapCreateDur 12.683µs SnapOpenDur 360.239371ms 2016-04-07T04:06:42.378-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22590000 QueuedCount 5397 2016-04-07T04:06:42.388-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31330000 2016-04-07T04:06:43.703-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.955µs CommitTime 905.857995ms TotalFlushTime 17m41.470019662s TotalCommitTime 1m31.266912024s 2016-04-07T04:06:43.703-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25107956, 22553978, 204 committed:true 2016-04-07T04:06:43.703-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25107956, 22553978, 204 committed:true) SnapCreateDur 917.370679ms SnapOpenDur 48.389µs 2016-04-07T04:06:43.726-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:43.75-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:43.774-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:43.798-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:43.8-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31340000 2016-04-07T04:06:43.823-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:43.849-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31350000 2016-04-07T04:06:43.889-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31360000 2016-04-07T04:06:43.938-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31370000 2016-04-07T04:06:43.981-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31380000 2016-04-07T04:06:44.09-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22600000 QueuedCount 48981 2016-04-07T04:06:44.885-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22610000 QueuedCount 38981 2016-04-07T04:06:45.599-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22620000 QueuedCount 28981 2016-04-07T04:06:46.266-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22630000 QueuedCount 18981 2016-04-07T04:06:48.024-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22640000 QueuedCount 8981 2016-04-07T04:06:48.693-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25213316, 22606658, 0 committed:false 2016-04-07T04:06:49.035-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25213316, 22606658, 0 committed:false) SnapCreateDur 28.527µs SnapOpenDur 342.297857ms 2016-04-07T04:06:49.046-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 615 2016-04-07T04:06:49.075-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22650000 QueuedCount 8668 2016-04-07T04:06:49.078-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31390000 2016-04-07T04:06:49.131-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31400000 2016-04-07T04:06:50.536-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 12.151µs CommitTime 784.614799ms TotalFlushTime 17m47.078420348s TotalCommitTime 1m32.051526823s 2016-04-07T04:06:50.536-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25232354, 22616177, 205 committed:true 2016-04-07T04:06:50.536-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25232354, 22616177, 205 committed:true) SnapCreateDur 796.578661ms SnapOpenDur 71.213µs 2016-04-07T04:06:50.561-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:50.582-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22660000 QueuedCount 9416 2016-04-07T04:06:50.585-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:50.609-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:50.622-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31410000 2016-04-07T04:06:50.633-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:50.656-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:50.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31420000 2016-04-07T04:06:50.717-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31430000 2016-04-07T04:06:50.768-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31440000 2016-04-07T04:06:50.829-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31450000 2016-04-07T04:06:51.443-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22670000 QueuedCount 34275 2016-04-07T04:06:52.158-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22680000 QueuedCount 24275 2016-04-07T04:06:52.836-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22690000 QueuedCount 14275 2016-04-07T04:06:53.533-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22700000 QueuedCount 4275 2016-04-07T04:06:53.83-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25322056, 22661028, 0 committed:false 2016-04-07T04:06:54.15-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25322056, 22661028, 0 committed:false) SnapCreateDur 12.379µs SnapOpenDur 320.362966ms 2016-04-07T04:06:54.211-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31460000 2016-04-07T04:06:54.555-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22710000 QueuedCount 3794 2016-04-07T04:06:55.483-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.302µs CommitTime 626.764054ms TotalFlushTime 17m50.981401226s TotalCommitTime 1m32.678290877s 2016-04-07T04:06:55.483-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25340780, 22670390, 206 committed:true 2016-04-07T04:06:55.483-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25340780, 22670390, 206 committed:true) SnapCreateDur 638.380915ms SnapOpenDur 35.6µs 2016-04-07T04:06:55.508-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:55.533-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:55.549-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31470000 2016-04-07T04:06:55.557-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:55.582-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:55.592-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31480000 2016-04-07T04:06:55.606-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:06:55.635-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31490000 2016-04-07T04:06:55.678-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31500000 2016-04-07T04:06:55.726-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31510000 2016-04-07T04:06:55.777-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31520000 2016-04-07T04:06:55.839-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31530000 2016-04-07T04:06:55.903-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31540000 2016-04-07T04:06:55.969-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22720000 QueuedCount 50000 2016-04-07T04:06:56.724-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22730000 QueuedCount 41912 2016-04-07T04:06:57.493-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22740000 QueuedCount 31912 2016-04-07T04:06:58.195-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22750000 QueuedCount 21912 2016-04-07T04:06:58.967-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22760000 QueuedCount 11912 2016-04-07T04:06:59.652-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22770000 QueuedCount 1912 2016-04-07T04:06:59.757-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25455250, 22727625, 0 committed:false 2016-04-07T04:07:00.111-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25455250, 22727625, 0 committed:false) SnapCreateDur 27.898µs SnapOpenDur 353.769762ms 2016-04-07T04:07:00.136-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31550000 2016-04-07T04:07:01.154-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22780000 QueuedCount 659 2016-04-07T04:07:02.43-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.926µs CommitTime 741.618781ms TotalFlushTime 17m56.729909163s TotalCommitTime 1m33.419909658s 2016-04-07T04:07:02.43-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25472490, 22736245, 207 committed:true 2016-04-07T04:07:02.43-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25472490, 22736245, 207 committed:true) SnapCreateDur 753.970923ms SnapOpenDur 37.293µs 2016-04-07T04:07:02.455-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:02.478-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:02.49-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31560000 2016-04-07T04:07:02.502-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:02.526-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:02.534-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31570000 2016-04-07T04:07:02.551-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:02.576-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31580000 2016-04-07T04:07:02.618-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31590000 2016-04-07T04:07:02.667-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31600000 2016-04-07T04:07:02.72-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31610000 2016-04-07T04:07:02.781-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31620000 2016-04-07T04:07:03.121-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22790000 QueuedCount 42889 2016-04-07T04:07:03.84-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22800000 QueuedCount 32889 2016-04-07T04:07:04.545-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22810000 QueuedCount 22889 2016-04-07T04:07:05.313-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22820000 QueuedCount 12889 2016-04-07T04:07:06-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22830000 QueuedCount 2889 2016-04-07T04:07:06.212-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25575320, 22787660, 0 committed:false 2016-04-07T04:07:06.556-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25575320, 22787660, 0 committed:false) SnapCreateDur 10.993µs SnapOpenDur 343.942967ms 2016-04-07T04:07:06.592-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31630000 2016-04-07T04:07:07.069-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22840000 QueuedCount 2406 2016-04-07T04:07:07.924-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.795µs CommitTime 657.890489ms TotalFlushTime 18m1.125398195s TotalCommitTime 1m34.077800147s 2016-04-07T04:07:07.924-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25594060, 22797030, 208 committed:true 2016-04-07T04:07:07.924-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25594060, 22797030, 208 committed:true) SnapCreateDur 669.842952ms SnapOpenDur 31.09µs 2016-04-07T04:07:07.951-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:07.955-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31640000 2016-04-07T04:07:07.976-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:07.999-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31650000 2016-04-07T04:07:08.001-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:08.029-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:08.041-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31660000 2016-04-07T04:07:08.053-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:08.083-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31670000 2016-04-07T04:07:08.13-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31680000 2016-04-07T04:07:08.496-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22850000 QueuedCount 45935 2016-04-07T04:07:09.231-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22860000 QueuedCount 35935 2016-04-07T04:07:09.948-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22870000 QueuedCount 25935 2016-04-07T04:07:10.642-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22880000 QueuedCount 15935 2016-04-07T04:07:11.319-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22890000 QueuedCount 5935 2016-04-07T04:07:12.876-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25699504, 22849752, 0 committed:false 2016-04-07T04:07:12.947-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8792662,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2435251418,"bucket-1:myindex:avg_ts_items_count":23927,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":7093597790,"bucket-1:myindex:delete_bytes":121583962,"bucket-1:myindex:disk_size":34946207744,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":79,"bucket-1:myindex:get_bytes":123555555,"bucket-1:myindex:insert_bytes":2396891027,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":208,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":23182124,"bucket-1:myindex:num_docs_pending":1423746,"bucket-1:myindex:num_docs_queued":9117193,"bucket-1:myindex:num_flush_queued":22895935,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":265,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1368024020,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"208 94077800147 -7199057131331061437","bucket-1:myindex:timings/storage_del":"2849752 70036447350 3920897755835983418","bucket-1:myindex:timings/storage_get":"22895935 155766826287 640297875229241457","bucket-1:myindex:timings/storage_handle_open":"208 4994163 126020398281","bucket-1:myindex:timings/storage_info":"947 8271531 202208784839","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"211 163277426 3672273710218096","bucket-1:myindex:timings/storage_meta_set":"208 411997985 107850042655270737","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"45699504 845303271216 -6668242022006611882","bucket-1:myindex:timings/storage_snapshot_close":"56 1859990357 66544271046074939","bucket-1:myindex:timings/storage_snapshot_create":"56 20302094915 -1974199785379312261","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":31688597,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":941,"memory_quota":31457280000,"memory_used":13686689792,"needs_restart":false,"num_connections":0,"timings/stats_response":"4122 587596959 101787661971027"} 2016-04-07T04:07:13.186-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 615 2016-04-07T04:07:13.216-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 615 2016-04-07T04:07:13.218-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25699504, 22849752, 0 committed:false) SnapCreateDur 10.3µs SnapOpenDur 341.739578ms 2016-04-07T04:07:13.252-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31690000 2016-04-07T04:07:13.451-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22900000 QueuedCount 1854 2016-04-07T04:07:14.292-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 21.925µs CommitTime 647.022956ms TotalFlushTime 18m6.41267574s TotalCommitTime 1m34.724823103s 2016-04-07T04:07:14.292-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25711128, 22855564, 209 committed:true 2016-04-07T04:07:14.292-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25711128, 22855564, 209 committed:true) SnapCreateDur 660.264571ms SnapOpenDur 44.091µs 2016-04-07T04:07:14.318-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:14.89-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:14.916-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:14.928-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31700000 2016-04-07T04:07:14.941-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:14.966-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:14.975-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31710000 2016-04-07T04:07:15.02-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31720000 2016-04-07T04:07:15.066-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31730000 2016-04-07T04:07:15.12-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31740000 2016-04-07T04:07:15.181-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31750000 2016-04-07T04:07:15.249-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31760000 2016-04-07T04:07:15.314-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31770000 2016-04-07T04:07:15.349-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22910000 QueuedCount 46820 2016-04-07T04:07:16.108-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22920000 QueuedCount 36820 2016-04-07T04:07:16.916-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22930000 QueuedCount 26820 2016-04-07T04:07:18.669-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22940000 QueuedCount 16820 2016-04-07T04:07:19.33-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22950000 QueuedCount 6820 2016-04-07T04:07:19.805-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25819250, 22909625, 0 committed:false 2016-04-07T04:07:20.164-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25819250, 22909625, 0 committed:false) SnapCreateDur 18.393µs SnapOpenDur 358.527713ms 2016-04-07T04:07:20.228-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31780000 2016-04-07T04:07:20.383-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22960000 QueuedCount 6574 2016-04-07T04:07:21.491-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.686µs CommitTime 641.78375ms TotalFlushTime 18m12.515111578s TotalCommitTime 1m35.366606853s 2016-04-07T04:07:21.493-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25838404, 22919202, 210 committed:true 2016-04-07T04:07:21.493-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25838404, 22919202, 210 committed:true) SnapCreateDur 654.225088ms SnapOpenDur 40.478µs 2016-04-07T04:07:21.518-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:21.542-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:21.547-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31790000 2016-04-07T04:07:21.566-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:21.589-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:21.593-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31800000 2016-04-07T04:07:21.612-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:21.641-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31810000 2016-04-07T04:07:21.689-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31820000 2016-04-07T04:07:21.742-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31830000 2016-04-07T04:07:21.774-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22970000 QueuedCount 46395 2016-04-07T04:07:22.508-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22980000 QueuedCount 36395 2016-04-07T04:07:23.242-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 22990000 QueuedCount 26395 2016-04-07T04:07:23.947-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23000000 QueuedCount 16395 2016-04-07T04:07:24.63-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23010000 QueuedCount 6395 2016-04-07T04:07:25.107-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25936488, 22968244, 0 committed:false 2016-04-07T04:07:25.446-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25936488, 22968244, 0 committed:false) SnapCreateDur 13.441µs SnapOpenDur 339.377153ms 2016-04-07T04:07:25.488-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31840000 2016-04-07T04:07:25.541-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31850000 2016-04-07T04:07:25.677-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23020000 QueuedCount 5201 2016-04-07T04:07:27.838-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.383µs CommitTime 677.319424ms TotalFlushTime 18m17.759866408s TotalCommitTime 1m36.043926277s 2016-04-07T04:07:27.839-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 25953804, 22976902, 211 committed:true 2016-04-07T04:07:27.839-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 25953804, 22976902, 211 committed:true) SnapCreateDur 688.107658ms SnapOpenDur 34.783µs 2016-04-07T04:07:27.863-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:27.887-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:27.908-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31860000 2016-04-07T04:07:27.911-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:27.936-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:27.952-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31870000 2016-04-07T04:07:27.961-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:27.996-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31880000 2016-04-07T04:07:28.041-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31890000 2016-04-07T04:07:28.093-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31900000 2016-04-07T04:07:28.152-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31910000 2016-04-07T04:07:28.208-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23030000 QueuedCount 48463 2016-04-07T04:07:28.213-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31920000 2016-04-07T04:07:28.276-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31930000 2016-04-07T04:07:28.928-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23040000 QueuedCount 38463 2016-04-07T04:07:29.631-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23050000 QueuedCount 28463 2016-04-07T04:07:31.391-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23060000 QueuedCount 18463 2016-04-07T04:07:32.036-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23070000 QueuedCount 8463 2016-04-07T04:07:32.618-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26058578, 23029289, 0 committed:false 2016-04-07T04:07:32.981-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26058578, 23029289, 0 committed:false) SnapCreateDur 13.886µs SnapOpenDur 363.21137ms 2016-04-07T04:07:33.014-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.150009% blocked 2016-04-07T04:07:33.015-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.153817% blocked 2016-04-07T04:07:33.015-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.151875% blocked 2016-04-07T04:07:33.015-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.149488% blocked 2016-04-07T04:07:33.016-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31940000 2016-04-07T04:07:33.067-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31950000 2016-04-07T04:07:33.108-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23080000 QueuedCount 10274 2016-04-07T04:07:33.13-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31960000 2016-04-07T04:07:33.788-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23090000 QueuedCount 274 2016-04-07T04:07:34.764-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.641µs CommitTime 956.212327ms TotalFlushTime 18m23.273355368s TotalCommitTime 1m37.000138604s 2016-04-07T04:07:34.764-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26081850, 23040925, 212 committed:true 2016-04-07T04:07:34.764-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26081850, 23040925, 212 committed:true) SnapCreateDur 967.492464ms SnapOpenDur 47.837µs 2016-04-07T04:07:34.789-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:34.812-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:34.831-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31970000 2016-04-07T04:07:34.835-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:34.86-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:34.88-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31980000 2016-04-07T04:07:34.884-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:34.931-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 31990000 2016-04-07T04:07:34.99-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32000000 2016-04-07T04:07:35.058-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32010000 2016-04-07T04:07:35.135-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32020000 2016-04-07T04:07:35.203-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32030000 2016-04-07T04:07:35.502-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23100000 QueuedCount 37139 2016-04-07T04:07:36.334-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23110000 QueuedCount 27139 2016-04-07T04:07:37.008-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23120000 QueuedCount 17139 2016-04-07T04:07:37.653-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23130000 QueuedCount 7139 2016-04-07T04:07:38.107-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26174110, 23087055, 0 committed:false 2016-04-07T04:07:38.441-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26174110, 23087055, 0 committed:false) SnapCreateDur 11.233µs SnapOpenDur 334.000309ms 2016-04-07T04:07:38.515-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32040000 2016-04-07T04:07:38.587-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32050000 2016-04-07T04:07:38.68-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23140000 QueuedCount 5822 2016-04-07T04:07:39.732-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.7µs CommitTime 645.107825ms TotalFlushTime 18m27.172136037s TotalCommitTime 1m37.645246429s 2016-04-07T04:07:39.732-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26191204, 23095602, 213 committed:true 2016-04-07T04:07:39.732-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26191204, 23095602, 213 committed:true) SnapCreateDur 655.882912ms SnapOpenDur 36.303µs 2016-04-07T04:07:39.756-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:39.781-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:39.792-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32060000 2016-04-07T04:07:39.805-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:39.829-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:39.84-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32070000 2016-04-07T04:07:39.853-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:39.892-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32080000 2016-04-07T04:07:39.938-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32090000 2016-04-07T04:07:39.991-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32100000 2016-04-07T04:07:40.045-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32110000 2016-04-07T04:07:40.11-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32120000 2016-04-07T04:07:40.129-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23150000 QueuedCount 50000 2016-04-07T04:07:40.172-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32130000 2016-04-07T04:07:40.238-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32140000 2016-04-07T04:07:40.865-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23160000 QueuedCount 41368 2016-04-07T04:07:41.578-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23170000 QueuedCount 31368 2016-04-07T04:07:42.264-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23180000 QueuedCount 21368 2016-04-07T04:07:42.964-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23190000 QueuedCount 11368 2016-04-07T04:07:43.631-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23200000 QueuedCount 1368 2016-04-07T04:07:43.732-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26300572, 23150286, 0 committed:false 2016-04-07T04:07:44.083-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26300572, 23150286, 0 committed:false) SnapCreateDur 17.823µs SnapOpenDur 351.386714ms 2016-04-07T04:07:45.332-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.909µs CommitTime 653.501461ms TotalFlushTime 18m31.664283607s TotalCommitTime 1m38.29874789s 2016-04-07T04:07:45.332-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26316566, 23158283, 214 committed:true 2016-04-07T04:07:45.332-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26316566, 23158283, 214 committed:true) SnapCreateDur 664.403111ms SnapOpenDur 43.393µs 2016-04-07T04:07:45.356-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:45.364-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23210000 QueuedCount 5566 2016-04-07T04:07:45.379-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:45.389-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32150000 2016-04-07T04:07:45.403-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:45.427-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:45.449-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32160000 2016-04-07T04:07:45.451-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:46.132-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23220000 QueuedCount 37533 2016-04-07T04:07:46.813-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23230000 QueuedCount 27533 2016-04-07T04:07:47.504-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23240000 QueuedCount 17533 2016-04-07T04:07:48.167-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23250000 QueuedCount 7533 2016-04-07T04:07:48.716-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26411142, 23205571, 0 committed:false 2016-04-07T04:07:49.05-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26411142, 23205571, 0 committed:false) SnapCreateDur 28.789µs SnapOpenDur 334.373001ms 2016-04-07T04:07:49.075-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:49.1-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:49.107-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32170000 2016-04-07T04:07:49.159-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32180000 2016-04-07T04:07:49.207-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32190000 2016-04-07T04:07:49.224-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23260000 QueuedCount 27316 2016-04-07T04:07:49.258-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32200000 2016-04-07T04:07:49.31-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32210000 2016-04-07T04:07:49.367-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32220000 2016-04-07T04:07:50.003-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23270000 QueuedCount 17316 2016-04-07T04:07:50.673-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23280000 QueuedCount 7316 2016-04-07T04:07:51.901-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.05µs CommitTime 744.786198ms TotalFlushTime 18m37.050951787s TotalCommitTime 1m39.043534088s 2016-04-07T04:07:51.901-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26469702, 23234851, 215 committed:true 2016-04-07T04:07:51.901-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26469702, 23234851, 215 committed:true) SnapCreateDur 755.198246ms SnapOpenDur 55.549µs 2016-04-07T04:07:51.925-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:51.949-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:51.95-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32230000 2016-04-07T04:07:51.973-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:51.995-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32240000 2016-04-07T04:07:51.997-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:52.02-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:52.04-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32250000 2016-04-07T04:07:52.613-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23290000 QueuedCount 43317 2016-04-07T04:07:53.377-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23300000 QueuedCount 33317 2016-04-07T04:07:54.029-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23310000 QueuedCount 23317 2016-04-07T04:07:54.656-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23320000 QueuedCount 13317 2016-04-07T04:07:55.282-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23330000 QueuedCount 3317 2016-04-07T04:07:55.494-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26560314, 23280157, 0 committed:false 2016-04-07T04:07:55.818-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26560314, 23280157, 0 committed:false) SnapCreateDur 18.108µs SnapOpenDur 324.228337ms 2016-04-07T04:07:55.869-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32260000 2016-04-07T04:07:57.176-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23340000 QueuedCount 2478 2016-04-07T04:07:58.04-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.843µs CommitTime 676.576744ms TotalFlushTime 18m42.102057955s TotalCommitTime 1m39.720110832s 2016-04-07T04:07:58.04-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26578320, 23289160, 216 committed:true 2016-04-07T04:07:58.04-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26578320, 23289160, 216 committed:true) SnapCreateDur 687.549154ms SnapOpenDur 41.587µs 2016-04-07T04:07:58.066-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:58.089-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:58.114-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:58.138-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:58.139-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32270000 2016-04-07T04:07:58.163-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:07:58.186-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32280000 2016-04-07T04:07:58.23-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32290000 2016-04-07T04:07:58.278-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32300000 2016-04-07T04:07:58.655-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23350000 QueuedCount 50000 2016-04-07T04:07:59.347-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23360000 QueuedCount 40826 2016-04-07T04:08:00.669-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23370000 QueuedCount 30826 2016-04-07T04:08:02.802-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23380000 QueuedCount 20826 2016-04-07T04:08:03.417-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23390000 QueuedCount 10826 2016-04-07T04:08:03.581-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 487 2016-04-07T04:08:03.611-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 487 2016-04-07T04:08:03.641-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 487 2016-04-07T04:08:03.671-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 487 2016-04-07T04:08:03.701-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 487 2016-04-07T04:08:03.731-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 487 2016-04-07T04:08:03.761-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 487 2016-04-07T04:08:03.791-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 487 2016-04-07T04:08:03.821-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 487 2016-04-07T04:08:03.851-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 487 2016-04-07T04:08:03.881-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 487 2016-04-07T04:08:03.911-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 487 2016-04-07T04:08:03.941-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 487 2016-04-07T04:08:03.971-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 487 2016-04-07T04:08:04.001-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 487 2016-04-07T04:08:04.031-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 487 2016-04-07T04:08:04.061-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 487 2016-04-07T04:08:04.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 487 2016-04-07T04:08:04.103-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23400000 QueuedCount 826 2016-04-07T04:08:04.121-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26693176, 23346588, 0 committed:false 2016-04-07T04:08:04.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 487 2016-04-07T04:08:04.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 487 2016-04-07T04:08:04.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 487 2016-04-07T04:08:04.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 487 2016-04-07T04:08:04.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 487 2016-04-07T04:08:04.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 487 2016-04-07T04:08:04.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 487 2016-04-07T04:08:04.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 487 2016-04-07T04:08:04.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 487 2016-04-07T04:08:04.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 487 2016-04-07T04:08:04.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 487 2016-04-07T04:08:04.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 487 2016-04-07T04:08:04.468-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26693176, 23346588, 0 committed:false) SnapCreateDur 16.843µs SnapOpenDur 346.97627ms 2016-04-07T04:08:04.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 487 2016-04-07T04:08:04.535-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32310000 2016-04-07T04:08:05.778-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 14.133µs CommitTime 705.450736ms TotalFlushTime 18m48.682272773s TotalCommitTime 1m40.425561568s 2016-04-07T04:08:05.778-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26710580, 23355290, 217 committed:true 2016-04-07T04:08:05.778-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26710580, 23355290, 217 committed:true) SnapCreateDur 715.610599ms SnapOpenDur 44.154µs 2016-04-07T04:08:05.802-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:05.805-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23410000 QueuedCount 8174 2016-04-07T04:08:05.826-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:05.845-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32320000 2016-04-07T04:08:05.849-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:05.872-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:05.891-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32330000 2016-04-07T04:08:05.895-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:05.938-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32340000 2016-04-07T04:08:05.99-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32350000 2016-04-07T04:08:06.551-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23420000 QueuedCount 34319 2016-04-07T04:08:07.241-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23430000 QueuedCount 24319 2016-04-07T04:08:07.887-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23440000 QueuedCount 14319 2016-04-07T04:08:08.52-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23450000 QueuedCount 4319 2016-04-07T04:08:08.8-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26798490, 23399245, 0 committed:false 2016-04-07T04:08:09.137-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26798490, 23399245, 0 committed:false) SnapCreateDur 45.462µs SnapOpenDur 337.408875ms 2016-04-07T04:08:09.162-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:09.186-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:09.19-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32360000 2016-04-07T04:08:09.21-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:09.234-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:09.258-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:09.3-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32370000 2016-04-07T04:08:09.345-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32380000 2016-04-07T04:08:09.475-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32390000 2016-04-07T04:08:09.57-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23460000 QueuedCount 50000 2016-04-07T04:08:09.667-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32400000 2016-04-07T04:08:11.368-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23470000 QueuedCount 41124 2016-04-07T04:08:11.979-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23480000 QueuedCount 31124 2016-04-07T04:08:12.589-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23490000 QueuedCount 21124 2016-04-07T04:08:12.948-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8891376,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3163374764,"bucket-1:myindex:avg_ts_items_count":34862,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":7200729279,"bucket-1:myindex:delete_bytes":146672355,"bucket-1:myindex:disk_size":39265587200,"bucket-1:myindex:flush_queue_size":15819,"bucket-1:myindex:frag_percent":81,"bucket-1:myindex:get_bytes":149048234,"bucket-1:myindex:insert_bytes":2458153425,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":217,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":23839212,"bucket-1:myindex:num_docs_pending":3646032,"bucket-1:myindex:num_docs_queued":9178760,"bucket-1:myindex:num_flush_queued":23511124,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":283,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":3359360775,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"217 100425561568 -2642076849960330714","bucket-1:myindex:timings/storage_del":"3439573 85157127131 4773418353407793001","bucket-1:myindex:timings/storage_get":"23495307 161099559618 640348880946262030","bucket-1:myindex:timings/storage_handle_open":"217 5193740 130568870118","bucket-1:myindex:timings/storage_info":"1001 8751899 211780736757","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"220 166629650 3673537038768052","bucket-1:myindex:timings/storage_meta_set":"217 415807325 107851689525190223","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"46879145 873866549631 -4811305312810422101","bucket-1:myindex:timings/storage_snapshot_close":"65 2234742215 82161675371906969","bucket-1:myindex:timings/storage_snapshot_create":"66 23731236741 -796991623301156599","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":32402500,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":911,"memory_quota":31457280000,"memory_used":15102902272,"needs_restart":false,"num_connections":0,"timings/stats_response":"4206 602807315 104630123725275"} 2016-04-07T04:08:13.248-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23500000 QueuedCount 11124 2016-04-07T04:08:13.94-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23510000 QueuedCount 1124 2016-04-07T04:08:14.036-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26910300, 23455150, 0 committed:false 2016-04-07T04:08:14.492-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26910300, 23455150, 0 committed:false) SnapCreateDur 13.15µs SnapOpenDur 456.773167ms 2016-04-07T04:08:14.578-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32410000 2016-04-07T04:08:14.631-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32420000 2016-04-07T04:08:15.101-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23520000 QueuedCount 1098 2016-04-07T04:08:16.2-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.41µs CommitTime 991.712001ms TotalFlushTime 18m57.170271898s TotalCommitTime 1m41.417273569s 2016-04-07T04:08:16.2-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 26929918, 23464959, 218 committed:true 2016-04-07T04:08:16.2-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 26929918, 23464959, 218 committed:true) SnapCreateDur 1.002658681s SnapOpenDur 44.921µs 2016-04-07T04:08:16.225-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:16.249-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:16.272-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:16.273-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32430000 2016-04-07T04:08:16.296-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:16.32-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:16.323-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32440000 2016-04-07T04:08:16.375-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32450000 2016-04-07T04:08:16.433-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32460000 2016-04-07T04:08:16.481-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32470000 2016-04-07T04:08:16.53-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32480000 2016-04-07T04:08:16.897-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23530000 QueuedCount 38497 2016-04-07T04:08:17.604-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23540000 QueuedCount 28497 2016-04-07T04:08:18.281-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23550000 QueuedCount 18497 2016-04-07T04:08:20.025-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23560000 QueuedCount 8497 2016-04-07T04:08:20.624-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27023280, 23511640, 0 committed:false 2016-04-07T04:08:20.956-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27023280, 23511640, 0 committed:false) SnapCreateDur 25.134µs SnapOpenDur 331.845382ms 2016-04-07T04:08:21.019-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32490000 2016-04-07T04:08:21.077-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23570000 QueuedCount 6495 2016-04-07T04:08:22.271-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.584µs CommitTime 688.394219ms TotalFlushTime 19m2.124775522s TotalCommitTime 1m42.105667788s 2016-04-07T04:08:22.271-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27039006, 23519503, 219 committed:true 2016-04-07T04:08:22.271-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27039006, 23519503, 219 committed:true) SnapCreateDur 700.325467ms SnapOpenDur 50.139µs 2016-04-07T04:08:22.297-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:22.321-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:22.328-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32500000 2016-04-07T04:08:22.345-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:22.371-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:22.396-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:22.408-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32510000 2016-04-07T04:08:22.454-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32520000 2016-04-07T04:08:22.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32530000 2016-04-07T04:08:22.541-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23580000 QueuedCount 50000 2016-04-07T04:08:22.553-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32540000 2016-04-07T04:08:23.253-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23590000 QueuedCount 43317 2016-04-07T04:08:23.969-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23600000 QueuedCount 33317 2016-04-07T04:08:24.659-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23610000 QueuedCount 23317 2016-04-07T04:08:25.416-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23620000 QueuedCount 13317 2016-04-07T04:08:26.096-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23630000 QueuedCount 3317 2016-04-07T04:08:26.293-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27150788, 23575394, 0 committed:false 2016-04-07T04:08:26.658-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27150788, 23575394, 0 committed:false) SnapCreateDur 22.344µs SnapOpenDur 365.067397ms 2016-04-07T04:08:26.739-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32550000 2016-04-07T04:08:27.171-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23640000 QueuedCount 371 2016-04-07T04:08:28.254-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 10.134µs CommitTime 1.059275012s TotalFlushTime 19m6.580579546s TotalCommitTime 1m43.1649428s 2016-04-07T04:08:28.254-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27164668, 23582334, 220 committed:true 2016-04-07T04:08:28.254-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27164668, 23582334, 220 committed:true) SnapCreateDur 1.071118572s SnapOpenDur 37.472µs 2016-04-07T04:08:28.279-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:28.304-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:28.323-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32560000 2016-04-07T04:08:28.328-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:28.352-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:28.373-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32570000 2016-04-07T04:08:28.376-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:28.427-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32580000 2016-04-07T04:08:28.488-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32590000 2016-04-07T04:08:28.548-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32600000 2016-04-07T04:08:28.597-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32610000 2016-04-07T04:08:28.96-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23650000 QueuedCount 38102 2016-04-07T04:08:29.652-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23660000 QueuedCount 28102 2016-04-07T04:08:30.452-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23670000 QueuedCount 18102 2016-04-07T04:08:31.137-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23680000 QueuedCount 8102 2016-04-07T04:08:31.709-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27258614, 23629307, 0 committed:false 2016-04-07T04:08:32.053-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27258614, 23629307, 0 committed:false) SnapCreateDur 11.089µs SnapOpenDur 343.275758ms 2016-04-07T04:08:32.096-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32620000 2016-04-07T04:08:32.143-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32630000 2016-04-07T04:08:32.185-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23690000 QueuedCount 6949 2016-04-07T04:08:33.297-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.405µs CommitTime 638.646797ms TotalFlushTime 19m10.552516342s TotalCommitTime 1m43.803589597s 2016-04-07T04:08:33.297-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27276020, 23638010, 221 committed:true 2016-04-07T04:08:33.297-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27276020, 23638010, 221 committed:true) SnapCreateDur 649.340992ms SnapOpenDur 40.224µs 2016-04-07T04:08:33.323-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:33.335-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32640000 2016-04-07T04:08:33.347-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:33.371-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:33.396-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:33.42-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:33.444-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32650000 2016-04-07T04:08:33.482-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32660000 2016-04-07T04:08:33.565-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23700000 QueuedCount 50000 2016-04-07T04:08:33.707-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32670000 2016-04-07T04:08:34.312-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23710000 QueuedCount 42286 2016-04-07T04:08:35.09-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23720000 QueuedCount 32286 2016-04-07T04:08:35.801-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23730000 QueuedCount 22286 2016-04-07T04:08:36.464-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23740000 QueuedCount 12286 2016-04-07T04:08:37.143-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23750000 QueuedCount 2286 2016-04-07T04:08:37.324-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27384970, 23692485, 0 committed:false 2016-04-07T04:08:37.701-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27384970, 23692485, 0 committed:false) SnapCreateDur 11.471µs SnapOpenDur 377.477027ms 2016-04-07T04:08:37.772-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32680000 2016-04-07T04:08:38.333-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23760000 QueuedCount 1388 2016-04-07T04:08:39.087-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.778µs CommitTime 648.247384ms TotalFlushTime 19m15.223318331s TotalCommitTime 1m44.451836981s 2016-04-07T04:08:39.088-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27402894, 23701447, 222 committed:true 2016-04-07T04:08:39.088-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27402894, 23701447, 222 committed:true) SnapCreateDur 659.971779ms SnapOpenDur 54.495µs 2016-04-07T04:08:39.113-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:39.138-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:39.139-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32690000 2016-04-07T04:08:39.162-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:39.182-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32700000 2016-04-07T04:08:39.187-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:39.211-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:39.226-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32710000 2016-04-07T04:08:39.27-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32720000 2016-04-07T04:08:39.777-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23770000 QueuedCount 39321 2016-04-07T04:08:40.637-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23780000 QueuedCount 29321 2016-04-07T04:08:42.13-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23790000 QueuedCount 19321 2016-04-07T04:08:42.79-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23800000 QueuedCount 9321 2016-04-07T04:08:43.354-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27497248, 23748624, 0 committed:false 2016-04-07T04:08:43.683-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27497248, 23748624, 0 committed:false) SnapCreateDur 23.154µs SnapOpenDur 329.183782ms 2016-04-07T04:08:43.725-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32730000 2016-04-07T04:08:43.794-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23810000 QueuedCount 7615 2016-04-07T04:08:45.921-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.664µs CommitTime 672.260513ms TotalFlushTime 19m20.969567476s TotalCommitTime 1m45.124097494s 2016-04-07T04:08:45.921-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27513572, 23756786, 223 committed:true 2016-04-07T04:08:45.921-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27513572, 23756786, 223 committed:true) SnapCreateDur 683.272797ms SnapOpenDur 54.856µs 2016-04-07T04:08:45.947-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:45.971-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:45.995-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:46.019-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:46.021-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32740000 2016-04-07T04:08:46.044-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:46.068-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32750000 2016-04-07T04:08:46.084-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23820000 QueuedCount 50000 2016-04-07T04:08:46.106-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32760000 2016-04-07T04:08:46.15-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32770000 2016-04-07T04:08:46.193-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32780000 2016-04-07T04:08:46.421-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32790000 2016-04-07T04:08:46.86-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23830000 QueuedCount 43762 2016-04-07T04:08:47.629-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23840000 QueuedCount 33762 2016-04-07T04:08:48.343-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23850000 QueuedCount 23762 2016-04-07T04:08:48.996-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23860000 QueuedCount 13762 2016-04-07T04:08:49.647-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23870000 QueuedCount 3762 2016-04-07T04:08:49.945-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27624160, 23812080, 0 committed:false 2016-04-07T04:08:50.297-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27624160, 23812080, 0 committed:false) SnapCreateDur 10.782µs SnapOpenDur 351.787212ms 2016-04-07T04:08:50.727-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23880000 QueuedCount 3928 2016-04-07T04:08:51.774-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 15.6µs CommitTime 750.188344ms TotalFlushTime 19m25.625548462s TotalCommitTime 1m45.874285838s 2016-04-07T04:08:51.774-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27644182, 23822091, 224 committed:true 2016-04-07T04:08:51.774-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27644182, 23822091, 224 committed:true) SnapCreateDur 761.771953ms SnapOpenDur 48.686µs 2016-04-07T04:08:51.8-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:51.824-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:51.838-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32800000 2016-04-07T04:08:51.849-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:51.874-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:51.886-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32810000 2016-04-07T04:08:51.898-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:51.933-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32820000 2016-04-07T04:08:51.988-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32830000 2016-04-07T04:08:52.042-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32840000 2016-04-07T04:08:52.091-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32850000 2016-04-07T04:08:52.141-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32860000 2016-04-07T04:08:53.422-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23890000 QueuedCount 47385 2016-04-07T04:08:54.184-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23900000 QueuedCount 37385 2016-04-07T04:08:54.981-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23910000 QueuedCount 27385 2016-04-07T04:08:55.746-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23920000 QueuedCount 17385 2016-04-07T04:08:56.465-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23930000 QueuedCount 7385 2016-04-07T04:08:56.983-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27749344, 23874672, 0 committed:false 2016-04-07T04:08:57.159-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:08:57.189-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:08:57.219-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:08:57.249-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:08:57.279-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:08:57.309-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:08:57.339-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:08:57.352-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27749344, 23874672, 0 committed:false) SnapCreateDur 22.477µs SnapOpenDur 369.027732ms 2016-04-07T04:08:57.369-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:08:57.399-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:08:57.429-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:08:57.459-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:08:57.489-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:08:57.519-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:08:57.549-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:08:57.568-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23940000 QueuedCount 3077 2016-04-07T04:08:57.579-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:08:57.609-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:08:57.639-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:08:57.669-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:08:57.699-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:08:57.729-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:08:57.759-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 649 2016-04-07T04:08:57.789-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 649 2016-04-07T04:08:57.819-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 649 2016-04-07T04:08:57.849-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 649 2016-04-07T04:08:57.879-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 649 2016-04-07T04:08:57.909-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 649 2016-04-07T04:08:57.939-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 649 2016-04-07T04:08:57.969-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 649 2016-04-07T04:08:57.999-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 649 2016-04-07T04:08:58.029-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 649 2016-04-07T04:08:58.059-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 649 2016-04-07T04:08:58.089-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 649 2016-04-07T04:08:58.119-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 649 2016-04-07T04:08:58.149-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 649 2016-04-07T04:08:58.179-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 649 2016-04-07T04:08:58.209-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 649 2016-04-07T04:08:58.239-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 649 2016-04-07T04:08:58.269-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 649 2016-04-07T04:08:58.299-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 649 2016-04-07T04:08:58.329-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 649 2016-04-07T04:08:58.359-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 649 2016-04-07T04:08:58.389-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 649 2016-04-07T04:08:58.419-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 649 2016-04-07T04:08:58.449-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 649 2016-04-07T04:08:58.479-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 649 2016-04-07T04:08:58.502-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.584µs CommitTime 654.41172ms TotalFlushTime 19m31.237698497s TotalCommitTime 1m46.528697558s 2016-04-07T04:08:58.502-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27760540, 23880270, 225 committed:true 2016-04-07T04:08:58.502-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27760540, 23880270, 225 committed:true) SnapCreateDur 666.126347ms SnapOpenDur 49.171µs 2016-04-07T04:08:58.509-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 649 2016-04-07T04:08:58.526-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:58.552-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:58.576-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:58.58-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32870000 2016-04-07T04:08:58.6-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:58.624-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:08:58.627-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32880000 2016-04-07T04:08:58.674-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32890000 2016-04-07T04:08:58.72-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32900000 2016-04-07T04:08:58.767-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32910000 2016-04-07T04:08:59.044-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23950000 QueuedCount 47140 2016-04-07T04:09:01.234-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23960000 QueuedCount 37140 2016-04-07T04:09:01.97-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23970000 QueuedCount 27140 2016-04-07T04:09:02.682-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23980000 QueuedCount 17140 2016-04-07T04:09:03.403-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 23990000 QueuedCount 7140 2016-04-07T04:09:03.802-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:09:03.832-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:09:03.862-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:09:03.885-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27866972, 23933486, 0 committed:false 2016-04-07T04:09:03.892-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:09:03.922-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:09:03.952-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:09:03.982-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:09:04.012-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:09:04.042-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:09:04.072-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:09:04.102-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:09:04.132-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:09:04.162-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:09:04.192-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:09:04.222-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:09:04.252-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:09:04.253-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27866972, 23933486, 0 committed:false) SnapCreateDur 9.861µs SnapOpenDur 367.484486ms 2016-04-07T04:09:04.306-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32920000 2016-04-07T04:09:04.487-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24000000 QueuedCount 7904 2016-04-07T04:09:05.88-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 21.216µs CommitTime 790.921446ms TotalFlushTime 19m37.366697948s TotalCommitTime 1m47.319619004s 2016-04-07T04:09:05.88-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27888116, 23944058, 226 committed:true 2016-04-07T04:09:05.88-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27888116, 23944058, 226 committed:true) SnapCreateDur 802.078948ms SnapOpenDur 49.782µs 2016-04-07T04:09:05.905-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:05.93-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:05.948-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32930000 2016-04-07T04:09:05.955-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:05.979-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:06.004-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:06.005-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32940000 2016-04-07T04:09:06.047-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24010000 QueuedCount 50000 2016-04-07T04:09:06.053-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32950000 2016-04-07T04:09:06.103-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32960000 2016-04-07T04:09:06.819-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24020000 QueuedCount 43234 2016-04-07T04:09:07.564-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24030000 QueuedCount 33234 2016-04-07T04:09:08.298-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24040000 QueuedCount 23234 2016-04-07T04:09:09.169-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24050000 QueuedCount 13234 2016-04-07T04:09:09.88-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24060000 QueuedCount 3234 2016-04-07T04:09:10.089-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 27997136, 23998568, 0 committed:false 2016-04-07T04:09:10.471-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 27997136, 23998568, 0 committed:false) SnapCreateDur 23.884µs SnapOpenDur 381.954734ms 2016-04-07T04:09:10.541-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32970000 2016-04-07T04:09:11.815-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.442µs CommitTime 787.139484ms TotalFlushTime 19m42.028776684s TotalCommitTime 1m48.106758488s 2016-04-07T04:09:11.815-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28010296, 24005148, 227 committed:true 2016-04-07T04:09:11.815-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28010296, 24005148, 227 committed:true) SnapCreateDur 799.330501ms SnapOpenDur 48.581µs 2016-04-07T04:09:11.838-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24070000 QueuedCount 4771 2016-04-07T04:09:11.839-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:11.863-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:11.887-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:11.911-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:11.936-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:11.972-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32980000 2016-04-07T04:09:12.026-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 32990000 2016-04-07T04:09:12.079-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33000000 2016-04-07T04:09:12.136-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33010000 2016-04-07T04:09:12.197-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33020000 2016-04-07T04:09:12.258-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33030000 2016-04-07T04:09:12.315-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33040000 2016-04-07T04:09:12.719-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24080000 QueuedCount 41097 2016-04-07T04:09:12.948-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8928305,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2545299244,"bucket-1:myindex:avg_ts_items_count":23298,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":7310284536,"bucket-1:myindex:delete_bytes":172075938,"bucket-1:myindex:disk_size":43725062144,"bucket-1:myindex:flush_queue_size":37408,"bucket-1:myindex:frag_percent":83,"bucket-1:myindex:get_bytes":174858984,"bucket-1:myindex:insert_bytes":2520185387,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":227,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":24435688,"bucket-1:myindex:num_docs_pending":6225721,"bucket-1:myindex:num_docs_queued":9152795,"bucket-1:myindex:num_flush_queued":24121097,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":303,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1343300952,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"227 108106758488 3453577146048877134","bucket-1:myindex:timings/storage_del":"4018720 98582053833 5600188355723545583","bucket-1:myindex:timings/storage_get":"24083691 166468876564 640402318143278164","bucket-1:myindex:timings/storage_handle_open":"227 5448044 137280167424","bucket-1:myindex:timings/storage_info":"1061 9274273 223699425921","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"230 170937109 3675429715757953","bucket-1:myindex:timings/storage_meta_set":"227 420004012 107853560580239782","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"48037439 902402304065 -2984107590062086941","bucket-1:myindex:timings/storage_snapshot_close":"76 2750548399 106877453501949529","bucket-1:myindex:timings/storage_snapshot_create":"76 27404325689 564029752857671857","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":33049402,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":883,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"4290 619485286 108019131622686"} 2016-04-07T04:09:13.465-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24090000 QueuedCount 31097 2016-04-07T04:09:14.204-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24100000 QueuedCount 21097 2016-04-07T04:09:15.484-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24110000 QueuedCount 11097 2016-04-07T04:09:16.323-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24120000 QueuedCount 1097 2016-04-07T04:09:16.441-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28111078, 24055539, 0 committed:false 2016-04-07T04:09:16.851-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28111078, 24055539, 0 committed:false) SnapCreateDur 13.965µs SnapOpenDur 410.047261ms 2016-04-07T04:09:16.902-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33050000 2016-04-07T04:09:17.628-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24130000 QueuedCount 2211 2016-04-07T04:09:18.476-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.552µs CommitTime 715.92443ms TotalFlushTime 19m47.477073771s TotalCommitTime 1m48.822682918s 2016-04-07T04:09:18.477-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28132960, 24066480, 228 committed:true 2016-04-07T04:09:18.477-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28132960, 24066480, 228 committed:true) SnapCreateDur 729.763994ms SnapOpenDur 50.364µs 2016-04-07T04:09:18.502-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:18.519-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33060000 2016-04-07T04:09:18.527-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:18.552-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:18.574-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33070000 2016-04-07T04:09:18.577-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:18.602-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:18.65-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33080000 2016-04-07T04:09:19.08-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24140000 QueuedCount 47672 2016-04-07T04:09:19.776-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24150000 QueuedCount 37672 2016-04-07T04:09:20.429-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24160000 QueuedCount 27672 2016-04-07T04:09:21.21-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24170000 QueuedCount 17672 2016-04-07T04:09:21.925-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24180000 QueuedCount 7672 2016-04-07T04:09:22.406-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28242024, 24121012, 0 committed:false 2016-04-07T04:09:22.769-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28242024, 24121012, 0 committed:false) SnapCreateDur 20.334µs SnapOpenDur 363.545234ms 2016-04-07T04:09:22.979-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24190000 QueuedCount 5481 2016-04-07T04:09:24.087-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 13.088µs CommitTime 711.676703ms TotalFlushTime 19m51.912673106s TotalCommitTime 1m49.534359621s 2016-04-07T04:09:24.087-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28257384, 24128692, 229 committed:true 2016-04-07T04:09:24.087-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28257384, 24128692, 229 committed:true) SnapCreateDur 723.050753ms SnapOpenDur 51.438µs 2016-04-07T04:09:24.111-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:24.134-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:24.158-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:24.166-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33090000 2016-04-07T04:09:24.183-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:24.209-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:24.223-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33100000 2016-04-07T04:09:24.27-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33110000 2016-04-07T04:09:24.325-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33120000 2016-04-07T04:09:24.503-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24200000 QueuedCount 42691 2016-04-07T04:09:25.417-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24210000 QueuedCount 32691 2016-04-07T04:09:27.745-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24220000 QueuedCount 22691 2016-04-07T04:09:28.455-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24230000 QueuedCount 12691 2016-04-07T04:09:29.372-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:09:29.378-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24240000 QueuedCount 2691 2016-04-07T04:09:29.402-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:09:29.432-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:09:29.462-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:09:29.492-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:09:29.522-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:09:29.552-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:09:29.582-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:09:29.612-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:09:29.613-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28350296, 24175148, 0 committed:false 2016-04-07T04:09:29.642-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:09:29.672-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:09:29.702-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:09:29.732-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:09:29.762-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:09:29.792-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:09:29.822-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:09:29.852-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:09:29.882-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:09:29.912-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:09:29.942-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:09:29.967-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28350296, 24175148, 0 committed:false) SnapCreateDur 12.559µs SnapOpenDur 353.464523ms 2016-04-07T04:09:29.972-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 649 2016-04-07T04:09:30.003-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.207326% blocked 2016-04-07T04:09:30.024-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33130000 2016-04-07T04:09:30.491-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24250000 QueuedCount 1901 2016-04-07T04:09:31.361-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.65µs CommitTime 666.918496ms TotalFlushTime 19m58.083430451s TotalCommitTime 1m50.201278117s 2016-04-07T04:09:31.361-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28368426, 24184213, 230 committed:true 2016-04-07T04:09:31.361-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28368426, 24184213, 230 committed:true) SnapCreateDur 678.308406ms SnapOpenDur 37.086µs 2016-04-07T04:09:31.386-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.204313% blocked 2016-04-07T04:09:31.387-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:31.405-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33140000 2016-04-07T04:09:31.414-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:31.439-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:31.458-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33150000 2016-04-07T04:09:31.465-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:31.513-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33160000 2016-04-07T04:09:31.56-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33170000 2016-04-07T04:09:31.616-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33180000 2016-04-07T04:09:31.631-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:31.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33190000 2016-04-07T04:09:31.811-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33200000 2016-04-07T04:09:32.024-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24260000 QueuedCount 50000 2016-04-07T04:09:32.137-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.204062% blocked 2016-04-07T04:09:32.274-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33210000 2016-04-07T04:09:32.865-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24270000 QueuedCount 44993 2016-04-07T04:09:33.76-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24280000 QueuedCount 34993 2016-04-07T04:09:36.239-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24290000 QueuedCount 24993 2016-04-07T04:09:36.896-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24300000 QueuedCount 14993 2016-04-07T04:09:37.382-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:09:37.412-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:09:37.442-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:09:37.472-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:09:37.502-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:09:37.532-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:09:37.54-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24310000 QueuedCount 4993 2016-04-07T04:09:37.562-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:09:37.592-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:09:37.622-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:09:37.652-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:09:37.682-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:09:37.712-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:09:37.742-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:09:37.772-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:09:37.802-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:09:37.832-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:09:37.862-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:09:37.892-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:09:37.922-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:09:37.952-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:09:37.96-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28492518, 24246259, 0 committed:false 2016-04-07T04:09:37.982-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 649 2016-04-07T04:09:38.012-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 649 2016-04-07T04:09:38.042-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 649 2016-04-07T04:09:38.072-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 649 2016-04-07T04:09:38.102-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 649 2016-04-07T04:09:38.132-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 649 2016-04-07T04:09:38.162-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 649 2016-04-07T04:09:38.192-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 649 2016-04-07T04:09:38.222-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 649 2016-04-07T04:09:38.252-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 649 2016-04-07T04:09:38.282-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 649 2016-04-07T04:09:38.312-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 649 2016-04-07T04:09:38.342-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 649 2016-04-07T04:09:38.343-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28492518, 24246259, 0 committed:false) SnapCreateDur 11.337µs SnapOpenDur 382.418716ms 2016-04-07T04:09:38.373-07:00 [Info] DATP[->dataport ":9105"] DATP -> Indexer 0.209239% blocked 2016-04-07T04:09:38.419-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33220000 2016-04-07T04:09:38.702-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24320000 QueuedCount 4262 2016-04-07T04:09:39.813-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.131µs CommitTime 773.919157ms TotalFlushTime 20m5.264248465s TotalCommitTime 1m50.975197274s 2016-04-07T04:09:39.813-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28510754, 24255377, 231 committed:true 2016-04-07T04:09:39.813-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28510754, 24255377, 231 committed:true) SnapCreateDur 784.962863ms SnapOpenDur 48.04µs 2016-04-07T04:09:39.837-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:39.862-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:39.883-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33230000 2016-04-07T04:09:39.885-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:39.909-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:39.931-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33240000 2016-04-07T04:09:39.933-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:39.982-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33250000 2016-04-07T04:09:40.035-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33260000 2016-04-07T04:09:40.336-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24330000 QueuedCount 45190 2016-04-07T04:09:41.591-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24340000 QueuedCount 35190 2016-04-07T04:09:43.105-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24350000 QueuedCount 25190 2016-04-07T04:09:44.752-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24360000 QueuedCount 15190 2016-04-07T04:09:45.091-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:09:45.121-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:09:45.151-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:09:45.181-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:09:45.211-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:09:45.241-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:09:45.271-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:09:45.301-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:09:45.331-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:09:45.361-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:09:45.391-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:09:45.402-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24370000 QueuedCount 5190 2016-04-07T04:09:45.421-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:09:45.451-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:09:45.481-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:09:45.511-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:09:45.541-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:09:45.571-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:09:45.601-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:09:45.631-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:09:45.661-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:09:45.691-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 649 2016-04-07T04:09:45.721-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 649 2016-04-07T04:09:45.751-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 649 2016-04-07T04:09:45.781-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 649 2016-04-07T04:09:45.811-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 649 2016-04-07T04:09:45.819-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28610994, 24305497, 0 committed:false 2016-04-07T04:09:45.841-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 649 2016-04-07T04:09:45.871-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 649 2016-04-07T04:09:45.901-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 649 2016-04-07T04:09:45.931-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 649 2016-04-07T04:09:45.961-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 649 2016-04-07T04:09:45.991-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 649 2016-04-07T04:09:46.021-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 649 2016-04-07T04:09:46.051-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 649 2016-04-07T04:09:46.081-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 649 2016-04-07T04:09:46.111-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 649 2016-04-07T04:09:46.141-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 649 2016-04-07T04:09:46.171-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 649 2016-04-07T04:09:46.201-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 649 2016-04-07T04:09:46.211-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28610994, 24305497, 0 committed:false) SnapCreateDur 11.135µs SnapOpenDur 392.52074ms 2016-04-07T04:09:46.231-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 649 2016-04-07T04:09:46.268-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33270000 2016-04-07T04:09:46.549-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24380000 QueuedCount 4046 2016-04-07T04:09:47.768-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 10.604µs CommitTime 900.949905ms TotalFlushTime 20m11.835649678s TotalCommitTime 1m51.876147179s 2016-04-07T04:09:47.769-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28628408, 24314204, 232 committed:true 2016-04-07T04:09:47.769-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28628408, 24314204, 232 committed:true) SnapCreateDur 912.853159ms SnapOpenDur 55.746µs 2016-04-07T04:09:47.793-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:47.818-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:47.822-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33280000 2016-04-07T04:09:47.844-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:47.868-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:47.874-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33290000 2016-04-07T04:09:47.926-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33300000 2016-04-07T04:09:47.936-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:47.976-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33310000 2016-04-07T04:09:48.032-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33320000 2016-04-07T04:09:48.174-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33330000 2016-04-07T04:09:48.279-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24390000 QueuedCount 50000 2016-04-07T04:09:49.128-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24400000 QueuedCount 45304 2016-04-07T04:09:49.911-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24410000 QueuedCount 35304 2016-04-07T04:09:50.765-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24420000 QueuedCount 25304 2016-04-07T04:09:51.524-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24430000 QueuedCount 15304 2016-04-07T04:09:52.35-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24440000 QueuedCount 5304 2016-04-07T04:09:52.688-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28749022, 24374511, 0 committed:false 2016-04-07T04:09:53.12-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28749022, 24374511, 0 committed:false) SnapCreateDur 15.96µs SnapOpenDur 432.022151ms 2016-04-07T04:09:53.21-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33340000 2016-04-07T04:09:53.535-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24450000 QueuedCount 2202 2016-04-07T04:09:54.377-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.981µs CommitTime 722.468796ms TotalFlushTime 20m17.174366631s TotalCommitTime 1m52.598615975s 2016-04-07T04:09:54.377-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28762604, 24381302, 233 committed:true 2016-04-07T04:09:54.377-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28762604, 24381302, 233 committed:true) SnapCreateDur 732.736514ms SnapOpenDur 41.741µs 2016-04-07T04:09:54.402-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:54.41-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33350000 2016-04-07T04:09:54.427-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:54.452-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:54.468-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33360000 2016-04-07T04:09:54.475-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:54.499-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:54.519-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33370000 2016-04-07T04:09:54.565-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33380000 2016-04-07T04:09:54.612-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33390000 2016-04-07T04:09:54.661-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33400000 2016-04-07T04:09:55.023-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24460000 QueuedCount 40894 2016-04-07T04:09:55.765-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24470000 QueuedCount 30894 2016-04-07T04:09:56.447-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24480000 QueuedCount 20894 2016-04-07T04:09:57.178-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24490000 QueuedCount 10894 2016-04-07T04:09:57.828-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24500000 QueuedCount 894 2016-04-07T04:09:57.852-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28858426, 24429213, 0 committed:false 2016-04-07T04:09:58.172-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28858426, 24429213, 0 committed:false) SnapCreateDur 22.54µs SnapOpenDur 320.58215ms 2016-04-07T04:09:59.55-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.659µs CommitTime 690.341392ms TotalFlushTime 20m21.237753953s TotalCommitTime 1m53.288957367s 2016-04-07T04:09:59.55-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 28875388, 24437694, 234 committed:true 2016-04-07T04:09:59.55-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 28875388, 24437694, 234 committed:true) SnapCreateDur 702.640078ms SnapOpenDur 40.931µs 2016-04-07T04:09:59.575-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:59.581-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24510000 QueuedCount 9355 2016-04-07T04:09:59.6-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:59.605-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33410000 2016-04-07T04:09:59.624-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:59.65-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:09:59.653-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33420000 2016-04-07T04:09:59.7-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33430000 2016-04-07T04:09:59.738-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33440000 2016-04-07T04:09:59.785-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33450000 2016-04-07T04:09:59.851-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:00.401-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24520000 QueuedCount 50000 2016-04-07T04:10:00.545-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33460000 2016-04-07T04:10:01.178-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24530000 QueuedCount 43321 2016-04-07T04:10:01.891-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24540000 QueuedCount 33321 2016-04-07T04:10:02.661-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24550000 QueuedCount 23321 2016-04-07T04:10:03.324-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24560000 QueuedCount 13321 2016-04-07T04:10:04.053-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24570000 QueuedCount 3321 2016-04-07T04:10:04.262-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29000964, 24500482, 0 committed:false 2016-04-07T04:10:04.639-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29000964, 24500482, 0 committed:false) SnapCreateDur 8.942µs SnapOpenDur 376.913865ms 2016-04-07T04:10:04.707-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33470000 2016-04-07T04:10:05.147-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24580000 QueuedCount 1406 2016-04-07T04:10:05.963-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.423µs CommitTime 709.632209ms TotalFlushTime 20m26.453507083s TotalCommitTime 1m53.998589576s 2016-04-07T04:10:05.964-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29016846, 24508423, 235 committed:true 2016-04-07T04:10:05.964-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29016846, 24508423, 235 committed:true) SnapCreateDur 720.139439ms SnapOpenDur 33.546µs 2016-04-07T04:10:05.989-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:06.014-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:06.024-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33480000 2016-04-07T04:10:06.039-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:06.063-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:06.073-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33490000 2016-04-07T04:10:06.087-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:06.125-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33500000 2016-04-07T04:10:06.17-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33510000 2016-04-07T04:10:06.218-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33520000 2016-04-07T04:10:06.742-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24590000 QueuedCount 45298 2016-04-07T04:10:07.521-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24600000 QueuedCount 35298 2016-04-07T04:10:08.267-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24610000 QueuedCount 25298 2016-04-07T04:10:08.986-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24620000 QueuedCount 15298 2016-04-07T04:10:09.658-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24630000 QueuedCount 5298 2016-04-07T04:10:10.033-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29122926, 24561463, 0 committed:false 2016-04-07T04:10:10.38-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29122926, 24561463, 0 committed:false) SnapCreateDur 10.724µs SnapOpenDur 346.940838ms 2016-04-07T04:10:10.44-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33530000 2016-04-07T04:10:10.497-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33540000 2016-04-07T04:10:10.711-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24640000 QueuedCount 2528 2016-04-07T04:10:11.662-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.624µs CommitTime 726.025674ms TotalFlushTime 20m30.984450043s TotalCommitTime 1m54.72461525s 2016-04-07T04:10:11.662-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29137156, 24568578, 236 committed:true 2016-04-07T04:10:11.662-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29137156, 24568578, 236 committed:true) SnapCreateDur 738.185476ms SnapOpenDur 37.788µs 2016-04-07T04:10:11.686-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:11.709-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:11.734-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:11.758-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:11.762-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33550000 2016-04-07T04:10:11.782-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:11.815-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33560000 2016-04-07T04:10:11.869-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33570000 2016-04-07T04:10:11.927-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33580000 2016-04-07T04:10:11.987-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33590000 2016-04-07T04:10:12.377-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24650000 QueuedCount 41587 2016-04-07T04:10:12.948-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8903932,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2374392836,"bucket-1:myindex:avg_ts_items_count":23502,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":7415117415,"bucket-1:myindex:delete_bytes":196448854,"bucket-1:myindex:disk_size":48671076352,"bucket-1:myindex:flush_queue_size":35122,"bucket-1:myindex:frag_percent":84,"bucket-1:myindex:get_bytes":199628704,"bucket-1:myindex:insert_bytes":2579700690,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":236,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":25025800,"bucket-1:myindex:num_docs_pending":5834721,"bucket-1:myindex:num_docs_queued":9139479,"bucket-1:myindex:num_flush_queued":24691587,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":321,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1282568321,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"236 114724615250 8357274088370215170","bucket-1:myindex:timings/storage_del":"4582295 112966319760 6857634845675419626","bucket-1:myindex:timings/storage_get":"24656466 171871429925 642172513908360545","bucket-1:myindex:timings/storage_handle_open":"236 5655234 142186658790","bucket-1:myindex:timings/storage_info":"1115 9765927 236983062135","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"239 174734109 3677090875220917","bucket-1:myindex:timings/storage_meta_set":"236 424443876 107855756628080934","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"49164588 931274233635 -1044954033687663837","bucket-1:myindex:timings/storage_snapshot_close":"85 3151239121 124895175748003333","bucket-1:myindex:timings/storage_snapshot_create":"85 30782047063 1840887697792382999","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":33595519,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":859,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"4432 646726781 113416943323095"} 2016-04-07T04:10:13.265-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24660000 QueuedCount 31587 2016-04-07T04:10:14.046-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24670000 QueuedCount 21587 2016-04-07T04:10:14.782-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24680000 QueuedCount 11587 2016-04-07T04:10:15.601-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24690000 QueuedCount 1587 2016-04-07T04:10:15.708-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29233750, 24616875, 0 committed:false 2016-04-07T04:10:16.081-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29233750, 24616875, 0 committed:false) SnapCreateDur 16.104µs SnapOpenDur 372.796116ms 2016-04-07T04:10:16.158-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33600000 2016-04-07T04:10:16.713-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24700000 QueuedCount 1528 2016-04-07T04:10:17.552-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.405µs CommitTime 724.999686ms TotalFlushTime 20m35.685758767s TotalCommitTime 1m55.449614936s 2016-04-07T04:10:17.552-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29253320, 24626660, 237 committed:true 2016-04-07T04:10:17.552-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29253320, 24626660, 237 committed:true) SnapCreateDur 736.544047ms SnapOpenDur 40.114µs 2016-04-07T04:10:17.577-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:17.599-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:17.6-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33610000 2016-04-07T04:10:17.628-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:17.649-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33620000 2016-04-07T04:10:17.652-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:17.696-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33630000 2016-04-07T04:10:17.735-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:17.741-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33640000 2016-04-07T04:10:17.788-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33650000 2016-04-07T04:10:17.84-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33660000 2016-04-07T04:10:17.891-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33670000 2016-04-07T04:10:18.366-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24710000 QueuedCount 50000 2016-04-07T04:10:18.401-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33680000 2016-04-07T04:10:19.088-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24720000 QueuedCount 43526 2016-04-07T04:10:19.837-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24730000 QueuedCount 33526 2016-04-07T04:10:20.524-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24740000 QueuedCount 23526 2016-04-07T04:10:21.209-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24750000 QueuedCount 13526 2016-04-07T04:10:21.944-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24760000 QueuedCount 3526 2016-04-07T04:10:22.276-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29375268, 24687634, 0 committed:false 2016-04-07T04:10:22.711-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29375268, 24687634, 0 committed:false) SnapCreateDur 9.795µs SnapOpenDur 435.858973ms 2016-04-07T04:10:22.8-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33690000 2016-04-07T04:10:23.155-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24770000 QueuedCount 1725 2016-04-07T04:10:24.389-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 62.455µs CommitTime 1.040307909s TotalFlushTime 20m40.926407472s TotalCommitTime 1m56.489922845s 2016-04-07T04:10:24.389-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29391384, 24695692, 238 committed:true 2016-04-07T04:10:24.389-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29391384, 24695692, 238 committed:true) SnapCreateDur 1.051996138s SnapOpenDur 42.487µs 2016-04-07T04:10:24.414-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:24.438-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:24.462-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:24.474-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33700000 2016-04-07T04:10:24.484-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:24.508-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:24.525-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33710000 2016-04-07T04:10:25.019-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24780000 QueuedCount 38624 2016-04-07T04:10:25.86-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24790000 QueuedCount 28624 2016-04-07T04:10:26.603-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24800000 QueuedCount 18624 2016-04-07T04:10:27.437-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24810000 QueuedCount 8624 2016-04-07T04:10:28.022-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29483608, 24741804, 0 committed:false 2016-04-07T04:10:28.339-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29483608, 24741804, 0 committed:false) SnapCreateDur 15.246µs SnapOpenDur 316.873068ms 2016-04-07T04:10:28.415-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33720000 2016-04-07T04:10:28.475-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24820000 QueuedCount 7460 2016-04-07T04:10:29.745-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.627µs CommitTime 730.387754ms TotalFlushTime 20m45.142163426s TotalCommitTime 1m57.220310599s 2016-04-07T04:10:29.745-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29501022, 24750511, 239 committed:true 2016-04-07T04:10:29.745-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29501022, 24750511, 239 committed:true) SnapCreateDur 741.212823ms SnapOpenDur 43.634µs 2016-04-07T04:10:29.769-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:29.783-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33730000 2016-04-07T04:10:29.793-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:29.819-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:29.839-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33740000 2016-04-07T04:10:29.843-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:29.87-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:29.895-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33750000 2016-04-07T04:10:29.918-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24830000 QueuedCount 50000 2016-04-07T04:10:29.948-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33760000 2016-04-07T04:10:30.012-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33770000 2016-04-07T04:10:30.059-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33780000 2016-04-07T04:10:30.119-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33790000 2016-04-07T04:10:30.667-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24840000 QueuedCount 46314 2016-04-07T04:10:31.486-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24850000 QueuedCount 36314 2016-04-07T04:10:32.267-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24860000 QueuedCount 26314 2016-04-07T04:10:32.936-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24870000 QueuedCount 16314 2016-04-07T04:10:33.605-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24880000 QueuedCount 6314 2016-04-07T04:10:34.07-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29616858, 24808429, 0 committed:false 2016-04-07T04:10:34.523-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29616858, 24808429, 0 committed:false) SnapCreateDur 26.242µs SnapOpenDur 452.829552ms 2016-04-07T04:10:34.558-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33800000 2016-04-07T04:10:34.769-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24890000 QueuedCount 6932 2016-04-07T04:10:36.039-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.899µs CommitTime 779.042504ms TotalFlushTime 20m50.095516593s TotalCommitTime 1m57.999353103s 2016-04-07T04:10:36.039-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29637740, 24818870, 240 committed:true 2016-04-07T04:10:36.039-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29637740, 24818870, 240 committed:true) SnapCreateDur 790.049679ms SnapOpenDur 41.129µs 2016-04-07T04:10:36.065-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:36.088-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33810000 2016-04-07T04:10:36.09-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:36.114-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:36.138-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33820000 2016-04-07T04:10:36.139-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:36.164-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:36.19-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33830000 2016-04-07T04:10:36.237-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33840000 2016-04-07T04:10:36.285-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33850000 2016-04-07T04:10:36.311-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24900000 QueuedCount 50000 2016-04-07T04:10:37.17-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24910000 QueuedCount 42120 2016-04-07T04:10:37.867-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24920000 QueuedCount 32120 2016-04-07T04:10:38.554-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24930000 QueuedCount 22120 2016-04-07T04:10:39.251-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24940000 QueuedCount 12120 2016-04-07T04:10:40.003-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24950000 QueuedCount 2120 2016-04-07T04:10:40.112-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29746424, 24873212, 0 committed:false 2016-04-07T04:10:40.463-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29746424, 24873212, 0 committed:false) SnapCreateDur 14.651µs SnapOpenDur 350.89737ms 2016-04-07T04:10:41.687-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.391µs CommitTime 719.449361ms TotalFlushTime 20m54.573689774s TotalCommitTime 1m58.718802464s 2016-04-07T04:10:41.687-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29758708, 24879354, 241 committed:true 2016-04-07T04:10:41.687-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29758708, 24879354, 241 committed:true) SnapCreateDur 730.174068ms SnapOpenDur 36.727µs 2016-04-07T04:10:41.71-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:41.733-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:41.757-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:41.763-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33860000 2016-04-07T04:10:41.781-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:41.805-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:41.81-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33870000 2016-04-07T04:10:41.854-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33880000 2016-04-07T04:10:41.864-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24960000 QueuedCount 45641 2016-04-07T04:10:41.9-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33890000 2016-04-07T04:10:41.946-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33900000 2016-04-07T04:10:44.605-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24970000 QueuedCount 35641 2016-04-07T04:10:45.314-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24980000 QueuedCount 25641 2016-04-07T04:10:46.031-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 24990000 QueuedCount 15641 2016-04-07T04:10:46.794-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25000000 QueuedCount 5641 2016-04-07T04:10:46.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:10:46.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:10:47.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:10:47.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:10:47.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:10:47.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:10:47.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:10:47.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:10:47.199-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29851810, 24925905, 0 committed:false 2016-04-07T04:10:47.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:10:47.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:10:47.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:10:47.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:10:47.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:10:47.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:10:47.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:10:47.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:10:47.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:10:47.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:10:47.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:10:47.527-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29851810, 24925905, 0 committed:false) SnapCreateDur 11.463µs SnapOpenDur 327.890883ms 2016-04-07T04:10:47.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:10:47.602-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33910000 2016-04-07T04:10:47.65-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33920000 2016-04-07T04:10:47.933-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25010000 QueuedCount 5480 2016-04-07T04:10:49.003-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.753µs CommitTime 640.564378ms TotalFlushTime 21m0.832915528s TotalCommitTime 1m59.359366842s 2016-04-07T04:10:49.003-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29871178, 24935589, 242 committed:true 2016-04-07T04:10:49.004-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29871178, 24935589, 242 committed:true) SnapCreateDur 651.958726ms SnapOpenDur 57.243µs 2016-04-07T04:10:49.028-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:49.053-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:49.078-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:49.081-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33930000 2016-04-07T04:10:49.103-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:49.128-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33940000 2016-04-07T04:10:49.167-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:49.173-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33950000 2016-04-07T04:10:49.223-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33960000 2016-04-07T04:10:49.306-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33970000 2016-04-07T04:10:49.423-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25020000 QueuedCount 50000 2016-04-07T04:10:49.566-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33980000 2016-04-07T04:10:50.242-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25030000 QueuedCount 46923 2016-04-07T04:10:51.042-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25040000 QueuedCount 36923 2016-04-07T04:10:51.824-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25050000 QueuedCount 26923 2016-04-07T04:10:52.653-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25060000 QueuedCount 16923 2016-04-07T04:10:53.333-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25070000 QueuedCount 6923 2016-04-07T04:10:53.817-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 29992054, 24996027, 0 committed:false 2016-04-07T04:10:54.197-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 29992054, 24996027, 0 committed:false) SnapCreateDur 36.355µs SnapOpenDur 380.304728ms 2016-04-07T04:10:54.269-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 33990000 2016-04-07T04:10:54.328-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34000000 2016-04-07T04:10:54.43-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25080000 QueuedCount 6806 2016-04-07T04:10:55.68-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.712µs CommitTime 736.449063ms TotalFlushTime 21m6.278104099s TotalCommitTime 2m0.095815905s 2016-04-07T04:10:55.68-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30011470, 25005735, 243 committed:true 2016-04-07T04:10:55.68-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30011470, 25005735, 243 committed:true) SnapCreateDur 747.005905ms SnapOpenDur 50.484µs 2016-04-07T04:10:55.706-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:55.73-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:55.754-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:55.763-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34010000 2016-04-07T04:10:55.777-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:55.799-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:10:55.812-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34020000 2016-04-07T04:10:55.858-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34030000 2016-04-07T04:10:55.91-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34040000 2016-04-07T04:10:55.994-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25090000 QueuedCount 40598 2016-04-07T04:10:56.914-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25100000 QueuedCount 30598 2016-04-07T04:10:57.744-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25110000 QueuedCount 20598 2016-04-07T04:10:58.562-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25120000 QueuedCount 10598 2016-04-07T04:10:59.429-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25130000 QueuedCount 598 2016-04-07T04:10:59.443-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30097652, 25048826, 0 committed:false 2016-04-07T04:10:59.819-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30097652, 25048826, 0 committed:false) SnapCreateDur 13.186µs SnapOpenDur 375.823159ms 2016-04-07T04:10:59.865-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34050000 2016-04-07T04:10:59.927-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34060000 2016-04-07T04:11:00.542-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25140000 QueuedCount 2078 2016-04-07T04:11:01.419-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.228µs CommitTime 753.701576ms TotalFlushTime 21m10.797063781s TotalCommitTime 2m0.849517481s 2016-04-07T04:11:01.419-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30120278, 25060139, 244 committed:true 2016-04-07T04:11:01.419-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30120278, 25060139, 244 committed:true) SnapCreateDur 763.824581ms SnapOpenDur 39.976µs 2016-04-07T04:11:01.445-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:01.462-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34070000 2016-04-07T04:11:01.47-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:01.494-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:01.514-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34080000 2016-04-07T04:11:01.519-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:01.557-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34090000 2016-04-07T04:11:01.599-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34100000 2016-04-07T04:11:01.729-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:01.796-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34110000 2016-04-07T04:11:02.132-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25150000 QueuedCount 50000 2016-04-07T04:11:02.145-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34120000 2016-04-07T04:11:02.893-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25160000 QueuedCount 45565 2016-04-07T04:11:03.728-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25170000 QueuedCount 35565 2016-04-07T04:11:04.467-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25180000 QueuedCount 25565 2016-04-07T04:11:05.184-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25190000 QueuedCount 15565 2016-04-07T04:11:05.9-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25200000 QueuedCount 5565 2016-04-07T04:11:06.325-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30245240, 25122620, 0 committed:false 2016-04-07T04:11:06.767-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30245240, 25122620, 0 committed:false) SnapCreateDur 20.656µs SnapOpenDur 442.122535ms 2016-04-07T04:11:06.852-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34130000 2016-04-07T04:11:07.146-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25210000 QueuedCount 5415 2016-04-07T04:11:08.557-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.739µs CommitTime 993.072773ms TotalFlushTime 21m16.379848551s TotalCommitTime 2m1.842590254s 2016-04-07T04:11:08.557-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30264620, 25132310, 245 committed:true 2016-04-07T04:11:08.557-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30264620, 25132310, 245 committed:true) SnapCreateDur 1.004419233s SnapOpenDur 44.213µs 2016-04-07T04:11:08.582-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:08.607-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:08.614-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34140000 2016-04-07T04:11:08.631-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:08.655-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:08.661-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34150000 2016-04-07T04:11:08.681-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:08.707-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34160000 2016-04-07T04:11:08.752-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34170000 2016-04-07T04:11:08.814-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34180000 2016-04-07T04:11:08.975-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25220000 QueuedCount 50000 2016-04-07T04:11:09.382-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34190000 2016-04-07T04:11:09.867-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25230000 QueuedCount 49603 2016-04-07T04:11:10.705-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25240000 QueuedCount 39603 2016-04-07T04:11:11.506-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25250000 QueuedCount 29603 2016-04-07T04:11:12.948-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8913229,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2861547500,"bucket-1:myindex:avg_ts_items_count":26590,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":7524681860,"bucket-1:myindex:delete_bytes":222042927,"bucket-1:myindex:disk_size":53131018240,"bucket-1:myindex:flush_queue_size":19619,"bucket-1:myindex:frag_percent":85,"bucket-1:myindex:get_bytes":225638071,"bucket-1:myindex:insert_bytes":2642197845,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":245,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":25670334,"bucket-1:myindex:num_docs_pending":5187284,"bucket-1:myindex:num_docs_queued":9142382,"bucket-1:myindex:num_flush_queued":25279603,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":339,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1789534943,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"245 121842590254 -4316684474941324558","bucket-1:myindex:timings/storage_del":"5176166 127711337010 7904131242507648162","bucket-1:myindex:timings/storage_get":"25259985 177774716065 645990976326980313","bucket-1:myindex:timings/storage_handle_open":"245 5845463 146266357033","bucket-1:myindex:timings/storage_info":"1169 10291187 252371578765","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"248 178127148 3678389583829366","bucket-1:myindex:timings/storage_meta_set":"245 428476440 107857593433221952","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"50352330 958773994024 973840531964791824","bucket-1:myindex:timings/storage_snapshot_close":"94 3564936640 144287277629357468","bucket-1:myindex:timings/storage_snapshot_create":"94 34236758633 3186761836177167403","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":34192832,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":840,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"4575 673257412 118448347419126"} 2016-04-07T04:11:12.96-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25260000 QueuedCount 19603 2016-04-07T04:11:14.614-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:11:14.644-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:11:14.674-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:11:14.704-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:11:14.734-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:11:14.764-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:11:14.794-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:11:14.824-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:11:14.854-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:11:14.884-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:11:14.914-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:11:14.944-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:11:14.974-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:11:15.004-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:11:15.034-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:11:15.064-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:11:15.094-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:11:15.124-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:11:15.154-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:11:15.184-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:11:15.214-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 649 2016-04-07T04:11:15.223-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25270000 QueuedCount 9603 2016-04-07T04:11:15.244-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 649 2016-04-07T04:11:15.274-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 649 2016-04-07T04:11:15.304-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 649 2016-04-07T04:11:15.334-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 649 2016-04-07T04:11:15.364-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 649 2016-04-07T04:11:15.394-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 649 2016-04-07T04:11:15.424-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 649 2016-04-07T04:11:15.454-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 649 2016-04-07T04:11:16.019-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 649 2016-04-07T04:11:16.024-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 649 2016-04-07T04:11:16.054-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 649 2016-04-07T04:11:16.084-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 649 2016-04-07T04:11:16.114-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 649 2016-04-07T04:11:16.144-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 649 2016-04-07T04:11:16.174-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 649 2016-04-07T04:11:16.204-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 649 2016-04-07T04:11:16.234-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 649 2016-04-07T04:11:16.264-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 649 2016-04-07T04:11:16.294-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 649 2016-04-07T04:11:16.324-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 649 2016-04-07T04:11:16.354-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 649 2016-04-07T04:11:16.384-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 649 2016-04-07T04:11:16.414-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 649 2016-04-07T04:11:16.444-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 649 2016-04-07T04:11:16.474-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 649 2016-04-07T04:11:16.5-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30390906, 25195453, 0 committed:false 2016-04-07T04:11:16.504-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 649 2016-04-07T04:11:16.534-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 649 2016-04-07T04:11:16.564-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 649 2016-04-07T04:11:16.594-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 649 2016-04-07T04:11:16.624-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 649 2016-04-07T04:11:16.654-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 649 2016-04-07T04:11:16.684-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 649 2016-04-07T04:11:16.714-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 649 2016-04-07T04:11:16.744-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 649 2016-04-07T04:11:16.774-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 649 2016-04-07T04:11:16.804-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 649 2016-04-07T04:11:16.834-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 649 2016-04-07T04:11:16.864-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 649 2016-04-07T04:11:16.894-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 649 2016-04-07T04:11:16.924-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 649 2016-04-07T04:11:16.954-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 649 2016-04-07T04:11:16.955-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30390906, 25195453, 0 committed:false) SnapCreateDur 17.55µs SnapOpenDur 455.199914ms 2016-04-07T04:11:17.019-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34200000 2016-04-07T04:11:17.286-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25280000 QueuedCount 9203 2016-04-07T04:11:21.418-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.648µs CommitTime 727.177261ms TotalFlushTime 21m27.932316966s TotalCommitTime 2m2.569767515s 2016-04-07T04:11:21.418-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30409838, 25204919, 246 committed:true 2016-04-07T04:11:21.418-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30409838, 25204919, 246 committed:true) SnapCreateDur 737.753973ms SnapOpenDur 41.494µs 2016-04-07T04:11:21.441-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:21.455-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25290000 QueuedCount 4747 2016-04-07T04:11:21.464-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:21.488-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:21.513-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:21.514-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34210000 2016-04-07T04:11:21.538-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:21.561-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34220000 2016-04-07T04:11:21.605-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34230000 2016-04-07T04:11:23.879-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25300000 QueuedCount 37070 2016-04-07T04:11:26.007-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25310000 QueuedCount 27070 2016-04-07T04:11:26.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:11:26.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:11:26.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:11:26.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:11:26.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:11:26.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:11:26.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:11:26.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:11:26.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:11:26.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:11:26.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:11:26.979-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25320000 QueuedCount 17070 2016-04-07T04:11:26.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:11:27.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:11:27.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:11:27.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:11:27.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:11:27.129-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:11:27.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:11:27.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:11:27.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:11:27.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 649 2016-04-07T04:11:27.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 649 2016-04-07T04:11:27.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 649 2016-04-07T04:11:27.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 649 2016-04-07T04:11:27.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 649 2016-04-07T04:11:27.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 649 2016-04-07T04:11:27.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 649 2016-04-07T04:11:27.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 649 2016-04-07T04:11:27.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 649 2016-04-07T04:11:27.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 649 2016-04-07T04:11:27.549-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 649 2016-04-07T04:11:27.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 649 2016-04-07T04:11:27.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 649 2016-04-07T04:11:27.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 649 2016-04-07T04:11:27.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 649 2016-04-07T04:11:27.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 649 2016-04-07T04:11:27.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 649 2016-04-07T04:11:27.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 649 2016-04-07T04:11:27.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 649 2016-04-07T04:11:27.797-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25330000 QueuedCount 7070 2016-04-07T04:11:27.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 649 2016-04-07T04:11:27.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 649 2016-04-07T04:11:27.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 649 2016-04-07T04:11:27.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 649 2016-04-07T04:11:27.939-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 649 2016-04-07T04:11:27.969-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 649 2016-04-07T04:11:28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 649 2016-04-07T04:11:28.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 649 2016-04-07T04:11:28.059-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 649 2016-04-07T04:11:28.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 649 2016-04-07T04:11:28.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 649 2016-04-07T04:11:28.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 649 2016-04-07T04:11:28.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 649 2016-04-07T04:11:28.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 649 2016-04-07T04:11:28.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 649 2016-04-07T04:11:28.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 649 2016-04-07T04:11:28.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 649 2016-04-07T04:11:28.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 649 2016-04-07T04:11:28.359-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 649 2016-04-07T04:11:28.372-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30503996, 25251998, 0 committed:false 2016-04-07T04:11:28.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 649 2016-04-07T04:11:28.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 649 2016-04-07T04:11:28.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 649 2016-04-07T04:11:28.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 649 2016-04-07T04:11:28.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 649 2016-04-07T04:11:28.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 649 2016-04-07T04:11:28.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 649 2016-04-07T04:11:28.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 649 2016-04-07T04:11:28.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 649 2016-04-07T04:11:28.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 649 2016-04-07T04:11:28.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 649 2016-04-07T04:11:28.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 649 2016-04-07T04:11:28.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 649 2016-04-07T04:11:28.757-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30503996, 25251998, 0 committed:false) SnapCreateDur 15.496µs SnapOpenDur 385.102005ms 2016-04-07T04:11:28.797-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34240000 2016-04-07T04:11:28.986-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25340000 QueuedCount 6695 2016-04-07T04:11:30.178-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 10.686µs CommitTime 705.401511ms TotalFlushTime 21m35.516419972s TotalCommitTime 2m3.275169026s 2016-04-07T04:11:30.178-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30522952, 25261476, 247 committed:true 2016-04-07T04:11:30.178-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30522952, 25261476, 247 committed:true) SnapCreateDur 715.958491ms SnapOpenDur 46.105µs 2016-04-07T04:11:30.203-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:30.227-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34250000 2016-04-07T04:11:30.228-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:30.252-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:30.269-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34260000 2016-04-07T04:11:30.276-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:30.3-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:30.316-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34270000 2016-04-07T04:11:30.358-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34280000 2016-04-07T04:11:30.408-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34290000 2016-04-07T04:11:30.463-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25350000 QueuedCount 50000 2016-04-07T04:11:30.465-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34300000 2016-04-07T04:11:30.531-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34310000 2016-04-07T04:11:30.6-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34320000 2016-04-07T04:11:31.316-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25360000 QueuedCount 43766 2016-04-07T04:11:32.188-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25370000 QueuedCount 33766 2016-04-07T04:11:32.951-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25380000 QueuedCount 23766 2016-04-07T04:11:33.768-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25390000 QueuedCount 13766 2016-04-07T04:11:34.481-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25400000 QueuedCount 3766 2016-04-07T04:11:34.8-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30635256, 25317628, 0 committed:false 2016-04-07T04:11:35.181-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30635256, 25317628, 0 committed:false) SnapCreateDur 11.553µs SnapOpenDur 381.335386ms 2016-04-07T04:11:35.232-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34330000 2016-04-07T04:11:35.634-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25410000 QueuedCount 4175 2016-04-07T04:11:36.713-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.716µs CommitTime 754.948137ms TotalFlushTime 21m40.811694761s TotalCommitTime 2m4.030117163s 2016-04-07T04:11:36.713-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30655702, 25327851, 248 committed:true 2016-04-07T04:11:36.713-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30655702, 25327851, 248 committed:true) SnapCreateDur 766.24059ms SnapOpenDur 65.979µs 2016-04-07T04:11:36.739-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:36.765-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:36.78-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34340000 2016-04-07T04:11:36.79-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:36.815-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:36.829-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34350000 2016-04-07T04:11:36.84-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:36.874-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34360000 2016-04-07T04:11:36.916-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34370000 2016-04-07T04:11:36.964-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34380000 2016-04-07T04:11:37.015-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34390000 2016-04-07T04:11:37.42-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25420000 QueuedCount 48512 2016-04-07T04:11:38.311-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25430000 QueuedCount 38512 2016-04-07T04:11:39.06-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25440000 QueuedCount 28512 2016-04-07T04:11:39.79-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25450000 QueuedCount 18512 2016-04-07T04:11:40.494-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25460000 QueuedCount 8512 2016-04-07T04:11:41.177-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30762644, 25381322, 0 committed:false 2016-04-07T04:11:41.536-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30762644, 25381322, 0 committed:false) SnapCreateDur 17.92µs SnapOpenDur 359.681127ms 2016-04-07T04:11:41.661-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25470000 QueuedCount 4535 2016-04-07T04:11:42.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:11:42.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:11:42.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:11:42.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:11:42.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:11:42.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:11:42.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:11:42.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:11:42.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:11:42.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:11:42.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:11:42.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:11:42.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:11:42.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:11:42.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:11:42.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:11:42.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:11:42.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:11:42.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:11:42.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:11:42.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 649 2016-04-07T04:11:42.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 649 2016-04-07T04:11:42.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 649 2016-04-07T04:11:42.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 649 2016-04-07T04:11:42.744-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.999µs CommitTime 753.006145ms TotalFlushTime 21m45.634123153s TotalCommitTime 2m4.783123308s 2016-04-07T04:11:42.744-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30774480, 25387240, 249 committed:true 2016-04-07T04:11:42.745-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30774480, 25387240, 249 committed:true) SnapCreateDur 764.263008ms SnapOpenDur 44.329µs 2016-04-07T04:11:42.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 649 2016-04-07T04:11:42.77-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:42.795-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:42.821-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:42.827-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34400000 2016-04-07T04:11:42.846-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:42.871-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:42.877-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34410000 2016-04-07T04:11:42.927-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34420000 2016-04-07T04:11:42.981-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34430000 2016-04-07T04:11:43.042-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34440000 2016-04-07T04:11:43.113-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34450000 2016-04-07T04:11:43.188-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34460000 2016-04-07T04:11:43.22-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25480000 QueuedCount 47239 2016-04-07T04:11:43.248-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34470000 2016-04-07T04:11:43.306-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34480000 2016-04-07T04:11:43.365-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34490000 2016-04-07T04:11:43.425-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34500000 2016-04-07T04:11:46.353-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25490000 QueuedCount 37239 2016-04-07T04:11:47.112-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25500000 QueuedCount 27239 2016-04-07T04:11:47.955-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25510000 QueuedCount 17239 2016-04-07T04:11:48.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:11:48.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:11:48.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:11:48.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:11:48.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:11:48.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:11:48.616-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:11:48.646-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:11:48.657-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25520000 QueuedCount 7239 2016-04-07T04:11:48.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:11:48.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:11:48.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:11:48.766-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:11:48.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:11:48.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:11:48.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:11:48.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:11:48.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:11:48.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:11:48.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:11:49.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:11:49.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 649 2016-04-07T04:11:49.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 649 2016-04-07T04:11:49.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 649 2016-04-07T04:11:49.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 649 2016-04-07T04:11:49.153-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30878142, 25439071, 0 committed:false 2016-04-07T04:11:49.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 649 2016-04-07T04:11:49.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 649 2016-04-07T04:11:49.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 649 2016-04-07T04:11:49.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 649 2016-04-07T04:11:49.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 649 2016-04-07T04:11:49.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 649 2016-04-07T04:11:49.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 649 2016-04-07T04:11:49.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 649 2016-04-07T04:11:49.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 649 2016-04-07T04:11:49.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 649 2016-04-07T04:11:49.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 649 2016-04-07T04:11:49.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 649 2016-04-07T04:11:49.502-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30878142, 25439071, 0 committed:false) SnapCreateDur 11.122µs SnapOpenDur 349.032509ms 2016-04-07T04:11:49.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 649 2016-04-07T04:11:49.753-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25530000 QueuedCount 10246 2016-04-07T04:11:50.523-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25540000 QueuedCount 246 2016-04-07T04:11:51.387-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.314µs CommitTime 756.792631ms TotalFlushTime 21m53.074770297s TotalCommitTime 2m5.539915939s 2016-04-07T04:11:51.387-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 30903774, 25451887, 250 committed:true 2016-04-07T04:11:51.387-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 30903774, 25451887, 250 committed:true) SnapCreateDur 767.950859ms SnapOpenDur 41.072µs 2016-04-07T04:11:51.413-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:51.443-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:51.471-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:51.482-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34510000 2016-04-07T04:11:51.498-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:51.534-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34520000 2016-04-07T04:11:51.579-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34530000 2016-04-07T04:11:51.635-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34540000 2016-04-07T04:11:51.684-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34550000 2016-04-07T04:11:51.749-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34560000 2016-04-07T04:11:51.81-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34570000 2016-04-07T04:11:51.97-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34580000 2016-04-07T04:11:52.195-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25550000 QueuedCount 50000 2016-04-07T04:11:52.442-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:11:52.979-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34590000 2016-04-07T04:11:53.153-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34600000 2016-04-07T04:11:53.193-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25560000 QueuedCount 50000 2016-04-07T04:11:53.211-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34610000 2016-04-07T04:11:55.354-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25570000 QueuedCount 44333 2016-04-07T04:11:58.19-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25580000 QueuedCount 34333 2016-04-07T04:11:58.502-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:11:58.532-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:11:58.562-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:11:58.592-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:11:58.622-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:11:58.652-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:11:58.682-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:11:58.712-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:11:58.742-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:11:58.772-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:11:58.802-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:11:58.832-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:11:58.862-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:11:58.892-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:11:58.922-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:11:58.952-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:11:58.982-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:11:59.012-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:11:59.035-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25590000 QueuedCount 24333 2016-04-07T04:11:59.042-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:11:59.072-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:11:59.102-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 649 2016-04-07T04:11:59.132-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 649 2016-04-07T04:11:59.162-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 649 2016-04-07T04:11:59.192-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 649 2016-04-07T04:11:59.222-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 649 2016-04-07T04:11:59.252-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 649 2016-04-07T04:11:59.282-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 649 2016-04-07T04:11:59.312-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 649 2016-04-07T04:11:59.342-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 649 2016-04-07T04:11:59.372-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 649 2016-04-07T04:11:59.402-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 649 2016-04-07T04:11:59.432-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 649 2016-04-07T04:11:59.462-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 649 2016-04-07T04:11:59.492-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 649 2016-04-07T04:11:59.522-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 649 2016-04-07T04:11:59.552-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 649 2016-04-07T04:11:59.582-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 649 2016-04-07T04:11:59.612-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 649 2016-04-07T04:11:59.642-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 649 2016-04-07T04:11:59.672-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 649 2016-04-07T04:11:59.702-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 649 2016-04-07T04:11:59.732-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 649 2016-04-07T04:11:59.762-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 649 2016-04-07T04:11:59.792-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 649 2016-04-07T04:11:59.822-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 649 2016-04-07T04:11:59.852-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 649 2016-04-07T04:11:59.862-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25600000 QueuedCount 14333 2016-04-07T04:11:59.882-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 649 2016-04-07T04:11:59.912-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 649 2016-04-07T04:11:59.942-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 649 2016-04-07T04:11:59.972-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 649 2016-04-07T04:12:00.002-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 649 2016-04-07T04:12:00.032-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 649 2016-04-07T04:12:00.062-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 649 2016-04-07T04:12:00.092-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 649 2016-04-07T04:12:00.122-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 649 2016-04-07T04:12:00.152-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 649 2016-04-07T04:12:00.182-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 649 2016-04-07T04:12:00.212-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 649 2016-04-07T04:12:00.242-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 649 2016-04-07T04:12:00.272-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 649 2016-04-07T04:12:00.302-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 649 2016-04-07T04:12:00.332-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 649 2016-04-07T04:12:00.362-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 649 2016-04-07T04:12:00.392-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 649 2016-04-07T04:12:00.422-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 649 2016-04-07T04:12:00.452-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 649 2016-04-07T04:12:00.482-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 649 2016-04-07T04:12:00.512-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 649 2016-04-07T04:12:00.542-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 649 2016-04-07T04:12:00.572-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 649 2016-04-07T04:12:00.602-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 649 2016-04-07T04:12:00.632-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 649 2016-04-07T04:12:00.662-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 649 2016-04-07T04:12:00.692-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 649 2016-04-07T04:12:00.693-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25610000 QueuedCount 4333 2016-04-07T04:12:00.722-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 649 2016-04-07T04:12:00.752-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 649 2016-04-07T04:12:00.782-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 649 2016-04-07T04:12:00.812-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 649 2016-04-07T04:12:00.842-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 649 2016-04-07T04:12:00.872-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 649 2016-04-07T04:12:00.902-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 649 2016-04-07T04:12:00.932-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 649 2016-04-07T04:12:00.962-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 649 2016-04-07T04:12:00.992-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 649 2016-04-07T04:12:01.022-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 649 2016-04-07T04:12:01.052-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 649 2016-04-07T04:12:01.082-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 649 2016-04-07T04:12:01.112-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 649 2016-04-07T04:12:01.123-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31049652, 25524826, 0 committed:false 2016-04-07T04:12:01.142-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 649 2016-04-07T04:12:01.172-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 649 2016-04-07T04:12:01.202-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 649 2016-04-07T04:12:01.232-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 649 2016-04-07T04:12:01.262-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 649 2016-04-07T04:12:01.292-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 649 2016-04-07T04:12:01.322-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 649 2016-04-07T04:12:01.352-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 649 2016-04-07T04:12:01.382-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 649 2016-04-07T04:12:01.412-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 649 2016-04-07T04:12:01.442-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 649 2016-04-07T04:12:01.472-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 649 2016-04-07T04:12:01.502-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 649 2016-04-07T04:12:01.532-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 649 2016-04-07T04:12:01.562-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 649 2016-04-07T04:12:01.592-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 649 2016-04-07T04:12:01.618-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31049652, 25524826, 0 committed:false) SnapCreateDur 12.142µs SnapOpenDur 494.511574ms 2016-04-07T04:12:01.622-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 649 2016-04-07T04:12:02.092-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25620000 QueuedCount 5290 2016-04-07T04:12:03.318-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.583µs CommitTime 772.693204ms TotalFlushTime 22m3.589554227s TotalCommitTime 2m6.312609143s 2016-04-07T04:12:03.318-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31071212, 25535606, 251 committed:true 2016-04-07T04:12:03.318-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31071212, 25535606, 251 committed:true) SnapCreateDur 783.505863ms SnapOpenDur 46.546µs 2016-04-07T04:12:03.345-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:03.37-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:03.393-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:03.415-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:03.439-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:03.695-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25630000 QueuedCount 39399 2016-04-07T04:12:04.565-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25640000 QueuedCount 29399 2016-04-07T04:12:05.302-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25650000 QueuedCount 19399 2016-04-07T04:12:06.029-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25660000 QueuedCount 9399 2016-04-07T04:12:06.746-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31158094, 25579047, 0 committed:false 2016-04-07T04:12:07.123-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31158094, 25579047, 0 committed:false) SnapCreateDur 29.32µs SnapOpenDur 377.743451ms 2016-04-07T04:12:07.188-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34620000 2016-04-07T04:12:07.249-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34630000 2016-04-07T04:12:07.258-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25670000 QueuedCount 10910 2016-04-07T04:12:07.311-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34640000 2016-04-07T04:12:07.374-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34650000 2016-04-07T04:12:07.442-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34660000 2016-04-07T04:12:07.514-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34670000 2016-04-07T04:12:07.59-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34680000 2016-04-07T04:12:07.672-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34690000 2016-04-07T04:12:08.042-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25680000 QueuedCount 910 2016-04-07T04:12:08.733-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 12.553µs CommitTime 661.06297ms TotalFlushTime 22m7.874498886s TotalCommitTime 2m6.973672113s 2016-04-07T04:12:08.733-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31180726, 25590363, 252 committed:true 2016-04-07T04:12:08.733-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31180726, 25590363, 252 committed:true) SnapCreateDur 672.29645ms SnapOpenDur 41.485µs 2016-04-07T04:12:08.761-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:08.786-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:08.811-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:08.836-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:08.862-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:08.923-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34700000 2016-04-07T04:12:08.977-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34710000 2016-04-07T04:12:09.021-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34720000 2016-04-07T04:12:09.074-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34730000 2016-04-07T04:12:09.123-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34740000 2016-04-07T04:12:09.198-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34750000 2016-04-07T04:12:09.457-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25690000 QueuedCount 49284 2016-04-07T04:12:10.33-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25700000 QueuedCount 39284 2016-04-07T04:12:11.074-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25710000 QueuedCount 29284 2016-04-07T04:12:11.879-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25720000 QueuedCount 19284 2016-04-07T04:12:12.597-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25730000 QueuedCount 9284 2016-04-07T04:12:12.949-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":9017633,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2812409901,"bucket-1:myindex:avg_ts_items_count":24371,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":7609774015,"bucket-1:myindex:delete_bytes":241917054,"bucket-1:myindex:disk_size":56668520448,"bucket-1:myindex:flush_queue_size":4067,"bucket-1:myindex:frag_percent":86,"bucket-1:myindex:get_bytes":245835515,"bucket-1:myindex:insert_bytes":2690727690,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":252,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":26139758,"bucket-1:myindex:num_docs_pending":4607722,"bucket-1:myindex:num_docs_queued":9252520,"bucket-1:myindex:num_flush_queued":25739284,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":353,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1609591044,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"252 126973672113 -546547346188338845","bucket-1:myindex:timings/storage_del":"5643784 143339413256 8854600823482895220","bucket-1:myindex:timings/storage_get":"25735218 182629820101 646044322620971089","bucket-1:myindex:timings/storage_handle_open":"252 6028103 151254287907","bucket-1:myindex:timings/storage_info":"1211 10664211 260384864237","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"255 180841165 3679465395162055","bucket-1:myindex:timings/storage_meta_set":"252 431631111 107859025907428913","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"51287566 989257823392 3042209295552565328","bucket-1:myindex:timings/storage_snapshot_close":"101 3878627822 158496081948556706","bucket-1:myindex:timings/storage_snapshot_create":"101 37038855147 4325706694529039231","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":34756917,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":826,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"4719 699657160 123475322805398"} 2016-04-07T04:12:13.223-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31295606, 25647803, 0 committed:false 2016-04-07T04:12:13.629-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31295606, 25647803, 0 committed:false) SnapCreateDur 20.067µs SnapOpenDur 405.723667ms 2016-04-07T04:12:13.744-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25740000 QueuedCount 11307 2016-04-07T04:12:14.545-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25750000 QueuedCount 1307 2016-04-07T04:12:15.466-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 10.415µs CommitTime 769.60654ms TotalFlushTime 22m13.332128196s TotalCommitTime 2m7.743278653s 2016-04-07T04:12:15.466-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31319250, 25659625, 253 committed:true 2016-04-07T04:12:15.466-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31319250, 25659625, 253 committed:true) SnapCreateDur 780.304325ms SnapOpenDur 52.034µs 2016-04-07T04:12:15.492-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:15.518-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:15.544-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:15.571-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:15.596-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:15.647-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34760000 2016-04-07T04:12:15.685-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34770000 2016-04-07T04:12:15.729-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34780000 2016-04-07T04:12:15.773-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34790000 2016-04-07T04:12:16.293-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25760000 QueuedCount 50000 2016-04-07T04:12:17.115-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25770000 QueuedCount 41828 2016-04-07T04:12:17.913-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25780000 QueuedCount 31828 2016-04-07T04:12:18.703-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25790000 QueuedCount 21828 2016-04-07T04:12:20.554-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25800000 QueuedCount 11828 2016-04-07T04:12:21.206-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 649 2016-04-07T04:12:21.236-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 649 2016-04-07T04:12:21.266-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 649 2016-04-07T04:12:21.296-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 649 2016-04-07T04:12:21.326-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 649 2016-04-07T04:12:21.356-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 649 2016-04-07T04:12:21.386-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 649 2016-04-07T04:12:21.416-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 649 2016-04-07T04:12:21.435-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25810000 QueuedCount 1828 2016-04-07T04:12:21.446-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 649 2016-04-07T04:12:21.476-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 649 2016-04-07T04:12:21.506-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 649 2016-04-07T04:12:21.536-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 649 2016-04-07T04:12:21.563-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31438312, 25719156, 0 committed:false 2016-04-07T04:12:21.566-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 649 2016-04-07T04:12:21.596-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 649 2016-04-07T04:12:21.626-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 649 2016-04-07T04:12:21.656-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 649 2016-04-07T04:12:21.686-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 649 2016-04-07T04:12:21.716-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 649 2016-04-07T04:12:21.746-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 649 2016-04-07T04:12:21.776-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 649 2016-04-07T04:12:21.806-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 649 2016-04-07T04:12:21.836-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 649 2016-04-07T04:12:21.866-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 649 2016-04-07T04:12:21.896-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 649 2016-04-07T04:12:21.926-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 649 2016-04-07T04:12:21.956-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 649 2016-04-07T04:12:21.96-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31438312, 25719156, 0 committed:false) SnapCreateDur 15.261µs SnapOpenDur 396.890742ms 2016-04-07T04:12:25.366-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 15.334µs CommitTime 801.236035ms TotalFlushTime 22m21.919557098s TotalCommitTime 2m8.544514688s 2016-04-07T04:12:25.366-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31447890, 25723945, 254 committed:true 2016-04-07T04:12:25.367-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31447890, 25723945, 254 committed:true) SnapCreateDur 813.102104ms SnapOpenDur 57.399µs 2016-04-07T04:12:25.389-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:25.415-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:25.439-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:25.465-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:25.475-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34800000 2016-04-07T04:12:25.491-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:25.522-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34810000 2016-04-07T04:12:25.566-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34820000 2016-04-07T04:12:25.622-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34830000 2016-04-07T04:12:25.663-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25820000 QueuedCount 50000 2016-04-07T04:12:25.67-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34840000 2016-04-07T04:12:25.722-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34850000 2016-04-07T04:12:25.777-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34860000 2016-04-07T04:12:25.835-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34870000 2016-04-07T04:12:25.9-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34880000 2016-04-07T04:12:25.97-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34890000 2016-04-07T04:12:26.047-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34900000 2016-04-07T04:12:26.587-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25830000 QueuedCount 40249 2016-04-07T04:12:27.326-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25840000 QueuedCount 30249 2016-04-07T04:12:28.08-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25850000 QueuedCount 20249 2016-04-07T04:12:28.941-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25860000 QueuedCount 10249 2016-04-07T04:12:29.667-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25870000 QueuedCount 249 2016-04-07T04:12:29.679-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31553252, 25776626, 0 committed:false 2016-04-07T04:12:30.063-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31553252, 25776626, 0 committed:false) SnapCreateDur 12.433µs SnapOpenDur 383.374795ms 2016-04-07T04:12:30.855-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25880000 QueuedCount 1192 2016-04-07T04:12:32.059-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.519µs CommitTime 1.078692206s TotalFlushTime 22m27.053725507s TotalCommitTime 2m9.623206894s 2016-04-07T04:12:32.059-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31574792, 25787396, 255 committed:true 2016-04-07T04:12:32.059-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31574792, 25787396, 255 committed:true) SnapCreateDur 1.090295645s SnapOpenDur 41.579µs 2016-04-07T04:12:32.083-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:32.107-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:32.131-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:32.156-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:32.161-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34910000 2016-04-07T04:12:32.181-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:32.206-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34920000 2016-04-07T04:12:32.249-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34930000 2016-04-07T04:12:32.288-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34940000 2016-04-07T04:12:32.333-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34950000 2016-04-07T04:12:32.384-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34960000 2016-04-07T04:12:32.438-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34970000 2016-04-07T04:12:32.496-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34980000 2016-04-07T04:12:32.551-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 34990000 2016-04-07T04:12:32.611-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35000000 2016-04-07T04:12:32.667-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35010000 2016-04-07T04:12:32.727-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35020000 2016-04-07T04:12:32.852-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25890000 QueuedCount 49221 2016-04-07T04:12:33.595-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25900000 QueuedCount 39221 2016-04-07T04:12:34.332-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25910000 QueuedCount 29221 2016-04-07T04:12:35.045-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25920000 QueuedCount 19221 2016-04-07T04:12:35.855-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25930000 QueuedCount 9221 2016-04-07T04:12:36.495-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31688952, 25844476, 0 committed:false 2016-04-07T04:12:36.874-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31688952, 25844476, 0 committed:false) SnapCreateDur 29.531µs SnapOpenDur 379.046453ms 2016-04-07T04:12:36.985-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25940000 QueuedCount 8851 2016-04-07T04:12:38.591-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.904µs CommitTime 980.469602ms TotalFlushTime 22m32.126839335s TotalCommitTime 2m10.603676496s 2016-04-07T04:12:38.591-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31707914, 25853957, 256 committed:true 2016-04-07T04:12:38.592-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31707914, 25853957, 256 committed:true) SnapCreateDur 992.404785ms SnapOpenDur 52.039µs 2016-04-07T04:12:38.616-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:38.64-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25950000 QueuedCount 15319 2016-04-07T04:12:38.642-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:38.66-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35030000 2016-04-07T04:12:38.665-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:38.689-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:38.697-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35040000 2016-04-07T04:12:38.713-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:38.736-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35050000 2016-04-07T04:12:38.774-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35060000 2016-04-07T04:12:38.81-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35070000 2016-04-07T04:12:38.86-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35080000 2016-04-07T04:12:38.916-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35090000 2016-04-07T04:12:39.607-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25960000 QueuedCount 45893 2016-04-07T04:12:40.346-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25970000 QueuedCount 35893 2016-04-07T04:12:41.081-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25980000 QueuedCount 25893 2016-04-07T04:12:41.759-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 25990000 QueuedCount 15893 2016-04-07T04:12:42.456-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26000000 QueuedCount 5893 2016-04-07T04:12:42.855-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31820068, 25910034, 0 committed:false 2016-04-07T04:12:43.228-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31820068, 25910034, 0 committed:false) SnapCreateDur 15.615µs SnapOpenDur 373.293054ms 2016-04-07T04:12:43.587-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26010000 QueuedCount 7477 2016-04-07T04:12:44.89-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.174µs CommitTime 676.044053ms TotalFlushTime 22m37.27633066s TotalCommitTime 2m11.279720549s 2016-04-07T04:12:44.89-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31842838, 25921419, 257 committed:true 2016-04-07T04:12:44.89-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31842838, 25921419, 257 committed:true) SnapCreateDur 686.243883ms SnapOpenDur 46.197µs 2016-04-07T04:12:44.913-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:44.937-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:44.962-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:44.988-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:44.99-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35100000 2016-04-07T04:12:45.012-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:45.027-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35110000 2016-04-07T04:12:45.065-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35120000 2016-04-07T04:12:45.068-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26020000 QueuedCount 50000 2016-04-07T04:12:45.104-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35130000 2016-04-07T04:12:45.154-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35140000 2016-04-07T04:12:45.198-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35150000 2016-04-07T04:12:45.255-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35160000 2016-04-07T04:12:45.309-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35170000 2016-04-07T04:12:45.369-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35180000 2016-04-07T04:12:45.424-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35190000 2016-04-07T04:12:45.861-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26030000 QueuedCount 44332 2016-04-07T04:12:46.666-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26040000 QueuedCount 34332 2016-04-07T04:12:47.405-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26050000 QueuedCount 24332 2016-04-07T04:12:48.089-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26060000 QueuedCount 14332 2016-04-07T04:12:48.779-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26070000 QueuedCount 4332 2016-04-07T04:12:49.064-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31954808, 25977404, 0 committed:false 2016-04-07T04:12:49.476-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31954808, 25977404, 0 committed:false) SnapCreateDur 10.003µs SnapOpenDur 411.799271ms 2016-04-07T04:12:49.929-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26080000 QueuedCount 2920 2016-04-07T04:12:50.899-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.303µs CommitTime 677.798211ms TotalFlushTime 22m42.107877663s TotalCommitTime 2m11.95751876s 2016-04-07T04:12:50.899-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 31971710, 25985855, 258 committed:true 2016-04-07T04:12:50.899-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 31971710, 25985855, 258 committed:true) SnapCreateDur 688.026769ms SnapOpenDur 51.78µs 2016-04-07T04:12:50.923-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:50.949-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:50.973-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:50.997-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:51.021-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:51.053-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35200000 2016-04-07T04:12:51.093-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35210000 2016-04-07T04:12:51.145-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35220000 2016-04-07T04:12:51.193-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35230000 2016-04-07T04:12:51.54-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26090000 QueuedCount 46872 2016-04-07T04:12:52.331-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26100000 QueuedCount 36872 2016-04-07T04:12:53.11-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26110000 QueuedCount 26872 2016-04-07T04:12:53.827-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26120000 QueuedCount 16872 2016-04-07T04:12:54.536-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26130000 QueuedCount 6872 2016-04-07T04:12:55.054-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32077864, 26038932, 0 committed:false 2016-04-07T04:12:55.405-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32077864, 26038932, 0 committed:false) SnapCreateDur 10.971µs SnapOpenDur 350.95973ms 2016-04-07T04:12:55.638-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26140000 QueuedCount 9483 2016-04-07T04:12:57.072-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.968µs CommitTime 721.255837ms TotalFlushTime 22m47.121697723s TotalCommitTime 2m12.678774597s 2016-04-07T04:12:57.074-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32102726, 26051363, 259 committed:true 2016-04-07T04:12:57.074-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32102726, 26051363, 259 committed:true) SnapCreateDur 732.770561ms SnapOpenDur 69.576µs 2016-04-07T04:12:57.1-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:57.107-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26150000 QueuedCount 14094 2016-04-07T04:12:57.125-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:57.15-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:57.15-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35240000 2016-04-07T04:12:57.174-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:57.19-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35250000 2016-04-07T04:12:57.232-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35260000 2016-04-07T04:12:57.267-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35270000 2016-04-07T04:12:57.307-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35280000 2016-04-07T04:12:57.351-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35290000 2016-04-07T04:12:57.389-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35300000 2016-04-07T04:12:57.435-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35310000 2016-04-07T04:12:57.482-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35320000 2016-04-07T04:12:57.539-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35330000 2016-04-07T04:12:57.576-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:12:57.598-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35340000 2016-04-07T04:12:57.942-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26160000 QueuedCount 50000 2016-04-07T04:12:58.822-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26170000 QueuedCount 41081 2016-04-07T04:12:59.543-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26180000 QueuedCount 31081 2016-04-07T04:13:00.319-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26190000 QueuedCount 21081 2016-04-07T04:13:00.991-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26200000 QueuedCount 11081 2016-04-07T04:13:02.229-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26210000 QueuedCount 1081 2016-04-07T04:13:02.328-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32224034, 26112017, 0 committed:false 2016-04-07T04:13:02.628-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 837 2016-04-07T04:13:02.658-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 837 2016-04-07T04:13:02.688-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 837 2016-04-07T04:13:02.718-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 837 2016-04-07T04:13:02.744-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32224034, 26112017, 0 committed:false) SnapCreateDur 13.112µs SnapOpenDur 415.16188ms 2016-04-07T04:13:02.748-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 837 2016-04-07T04:13:03.931-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.353µs CommitTime 773.073982ms TotalFlushTime 22m52.687785138s TotalCommitTime 2m13.451848579s 2016-04-07T04:13:03.931-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32233902, 26116951, 260 committed:true 2016-04-07T04:13:03.931-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32233902, 26116951, 260 committed:true) SnapCreateDur 783.965925ms SnapOpenDur 38.025µs 2016-04-07T04:13:03.954-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:03.978-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:04.002-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:04.027-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:04.052-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:04.097-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35350000 2016-04-07T04:13:04.138-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35360000 2016-04-07T04:13:04.191-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35370000 2016-04-07T04:13:04.233-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26220000 QueuedCount 50000 2016-04-07T04:13:04.237-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35380000 2016-04-07T04:13:04.295-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35390000 2016-04-07T04:13:04.356-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35400000 2016-04-07T04:13:05.219-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26230000 QueuedCount 42451 2016-04-07T04:13:05.989-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26240000 QueuedCount 32451 2016-04-07T04:13:06.776-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26250000 QueuedCount 22451 2016-04-07T04:13:07.536-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26260000 QueuedCount 12451 2016-04-07T04:13:08.377-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26270000 QueuedCount 2451 2016-04-07T04:13:08.572-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32344768, 26172384, 0 committed:false 2016-04-07T04:13:08.998-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32344768, 26172384, 0 committed:false) SnapCreateDur 17.213µs SnapOpenDur 426.309835ms 2016-04-07T04:13:09.612-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26280000 QueuedCount 1569 2016-04-07T04:13:10.708-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.892µs CommitTime 883.626161ms TotalFlushTime 22m58.053592067s TotalCommitTime 2m14.33547474s 2016-04-07T04:13:10.709-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32362734, 26181367, 261 committed:true 2016-04-07T04:13:10.709-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32362734, 26181367, 261 committed:true) SnapCreateDur 895.375228ms SnapOpenDur 51.692µs 2016-04-07T04:13:10.733-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:10.757-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:10.781-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:10.806-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:10.829-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35410000 2016-04-07T04:13:10.831-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:10.869-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35420000 2016-04-07T04:13:10.904-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35430000 2016-04-07T04:13:10.944-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35440000 2016-04-07T04:13:10.988-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35450000 2016-04-07T04:13:11.027-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35460000 2016-04-07T04:13:11.493-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26290000 QueuedCount 50000 2016-04-07T04:13:12.318-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26300000 QueuedCount 40490 2016-04-07T04:13:12.949-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":9128745,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2786944228,"bucket-1:myindex:avg_ts_items_count":24754,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":7714347329,"bucket-1:myindex:delete_bytes":266343677,"bucket-1:myindex:disk_size":61269106688,"bucket-1:myindex:flush_queue_size":32475,"bucket-1:myindex:frag_percent":87,"bucket-1:myindex:get_bytes":270660576,"bucket-1:myindex:insert_bytes":2750374095,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":261,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":26749716,"bucket-1:myindex:num_docs_pending":3323427,"bucket-1:myindex:num_docs_queued":9926857,"bucket-1:myindex:num_flush_queued":26340490,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":371,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1710232284,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"261 134335474740 5627718112623110964","bucket-1:myindex:timings/storage_del":"6207398 158045618504 -8546964932891067576","bucket-1:myindex:timings/storage_get":"26308016 188204882708 646477001737188562","bucket-1:myindex:timings/storage_handle_open":"261 6262186 157585756584","bucket-1:myindex:timings/storage_info":"1265 11196563 276053246987","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"264 184157281 3680704153030897","bucket-1:myindex:timings/storage_meta_set":"261 436088936 107861288578469512","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"52414794 1016827521895 5036190409070898965","bucket-1:myindex:timings/storage_snapshot_close":"110 4288346789 177181164108908045","bucket-1:myindex:timings/storage_snapshot_create":"110 40580772594 5724188000357489900","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":35469235,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":808,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"4863 727098832 128820149060236"} 2016-04-07T04:13:13.098-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26310000 QueuedCount 30490 2016-04-07T04:13:13.797-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26320000 QueuedCount 20490 2016-04-07T04:13:14.577-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26330000 QueuedCount 10490 2016-04-07T04:13:15.335-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26340000 QueuedCount 490 2016-04-07T04:13:15.352-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32478784, 26239392, 0 committed:false 2016-04-07T04:13:15.743-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32478784, 26239392, 0 committed:false) SnapCreateDur 17.279µs SnapOpenDur 391.302065ms 2016-04-07T04:13:16.989-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26350000 QueuedCount 834 2016-04-07T04:13:17.84-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.987µs CommitTime 819.736677ms TotalFlushTime 23m3.863451566s TotalCommitTime 2m15.155211417s 2016-04-07T04:13:17.84-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32499144, 26249572, 262 committed:true 2016-04-07T04:13:17.84-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32499144, 26249572, 262 committed:true) SnapCreateDur 831.442528ms SnapOpenDur 40.168µs 2016-04-07T04:13:17.865-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:17.89-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:17.915-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:17.917-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35470000 2016-04-07T04:13:17.94-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:17.965-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:18.614-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26360000 QueuedCount 42950 2016-04-07T04:13:19.403-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26370000 QueuedCount 32950 2016-04-07T04:13:20.243-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26380000 QueuedCount 22950 2016-04-07T04:13:21.108-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26390000 QueuedCount 12950 2016-04-07T04:13:21.797-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26400000 QueuedCount 2950 2016-04-07T04:13:22.01-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32601766, 26300883, 0 committed:false 2016-04-07T04:13:22.376-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32601766, 26300883, 0 committed:false) SnapCreateDur 21.737µs SnapOpenDur 365.221707ms 2016-04-07T04:13:22.461-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35480000 2016-04-07T04:13:22.508-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35490000 2016-04-07T04:13:23.546-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.407µs CommitTime 706.288499ms TotalFlushTime 23m8.410648904s TotalCommitTime 2m15.861499916s 2016-04-07T04:13:23.547-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32613984, 26306992, 263 committed:true 2016-04-07T04:13:23.547-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32613984, 26306992, 263 committed:true) SnapCreateDur 717.131482ms SnapOpenDur 53.434µs 2016-04-07T04:13:23.57-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:23.586-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26410000 QueuedCount 5204 2016-04-07T04:13:23.598-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:23.623-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:23.647-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:23.661-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35500000 2016-04-07T04:13:23.708-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35510000 2016-04-07T04:13:23.753-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35520000 2016-04-07T04:13:23.807-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35530000 2016-04-07T04:13:23.855-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35540000 2016-04-07T04:13:23.908-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35550000 2016-04-07T04:13:23.959-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35560000 2016-04-07T04:13:23.978-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:24.006-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35570000 2016-04-07T04:13:24.422-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26420000 QueuedCount 50000 2016-04-07T04:13:25.293-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26430000 QueuedCount 44605 2016-04-07T04:13:26.082-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26440000 QueuedCount 34605 2016-04-07T04:13:26.901-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26450000 QueuedCount 24605 2016-04-07T04:13:27.626-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26460000 QueuedCount 14605 2016-04-07T04:13:28.324-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26470000 QueuedCount 4605 2016-04-07T04:13:28.66-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32742772, 26371386, 0 committed:false 2016-04-07T04:13:29.076-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32742772, 26371386, 0 committed:false) SnapCreateDur 11.637µs SnapOpenDur 415.808059ms 2016-04-07T04:13:29.493-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26480000 QueuedCount 1639 2016-04-07T04:13:30.444-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.518µs CommitTime 833.562216ms TotalFlushTime 23m13.941961432s TotalCommitTime 2m16.695062132s 2016-04-07T04:13:30.444-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32756620, 26378310, 264 committed:true 2016-04-07T04:13:30.444-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32756620, 26378310, 264 committed:true) SnapCreateDur 844.455625ms SnapOpenDur 43.61µs 2016-04-07T04:13:30.466-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:30.491-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:30.515-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:30.534-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35580000 2016-04-07T04:13:30.539-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:30.564-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:30.571-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35590000 2016-04-07T04:13:30.607-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35600000 2016-04-07T04:13:30.644-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35610000 2016-04-07T04:13:30.685-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35620000 2016-04-07T04:13:31.203-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26490000 QueuedCount 44168 2016-04-07T04:13:32.084-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26500000 QueuedCount 34168 2016-04-07T04:13:32.857-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26510000 QueuedCount 24168 2016-04-07T04:13:33.607-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26520000 QueuedCount 14168 2016-04-07T04:13:34.335-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26530000 QueuedCount 4168 2016-04-07T04:13:34.635-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32860028, 26430014, 0 committed:false 2016-04-07T04:13:35.008-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32860028, 26430014, 0 committed:false) SnapCreateDur 9.381µs SnapOpenDur 373.053579ms 2016-04-07T04:13:35.075-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35630000 2016-04-07T04:13:35.12-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35640000 2016-04-07T04:13:35.168-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35650000 2016-04-07T04:13:35.443-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26540000 QueuedCount 3793 2016-04-07T04:13:36.51-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.143µs CommitTime 765.180959ms TotalFlushTime 23m18.777342244s TotalCommitTime 2m17.460243091s 2016-04-07T04:13:36.51-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32878984, 26439492, 265 committed:true 2016-04-07T04:13:36.51-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32878984, 26439492, 265 committed:true) SnapCreateDur 775.739519ms SnapOpenDur 39.743µs 2016-04-07T04:13:36.535-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:36.56-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:36.585-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:36.609-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:36.633-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:36.65-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35660000 2016-04-07T04:13:36.693-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35670000 2016-04-07T04:13:36.748-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35680000 2016-04-07T04:13:37.054-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26550000 QueuedCount 46113 2016-04-07T04:13:37.831-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26560000 QueuedCount 36113 2016-04-07T04:13:38.559-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26570000 QueuedCount 26113 2016-04-07T04:13:39.384-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26580000 QueuedCount 16113 2016-04-07T04:13:40.094-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26590000 QueuedCount 6113 2016-04-07T04:13:40.502-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32982096, 26491048, 0 committed:false 2016-04-07T04:13:40.858-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32982096, 26491048, 0 committed:false) SnapCreateDur 8.11µs SnapOpenDur 356.022463ms 2016-04-07T04:13:41.2-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26600000 QueuedCount 1252 2016-04-07T04:13:42.128-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.672µs CommitTime 775.490653ms TotalFlushTime 23m23.173254037s TotalCommitTime 2m18.235733744s 2016-04-07T04:13:42.128-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 32992210, 26496105, 266 committed:true 2016-04-07T04:13:42.128-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 32992210, 26496105, 266 committed:true) SnapCreateDur 786.948169ms SnapOpenDur 40.417µs 2016-04-07T04:13:42.151-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:42.176-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:42.191-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35690000 2016-04-07T04:13:42.2-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:42.225-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:42.233-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35700000 2016-04-07T04:13:42.249-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:42.278-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35710000 2016-04-07T04:13:42.319-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35720000 2016-04-07T04:13:42.357-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35730000 2016-04-07T04:13:42.407-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35740000 2016-04-07T04:13:42.45-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35750000 2016-04-07T04:13:42.492-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35760000 2016-04-07T04:13:42.54-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35770000 2016-04-07T04:13:42.956-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26610000 QueuedCount 44938 2016-04-07T04:13:43.783-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26620000 QueuedCount 34938 2016-04-07T04:13:44.593-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26630000 QueuedCount 24938 2016-04-07T04:13:45.357-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26640000 QueuedCount 14938 2016-04-07T04:13:46.195-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26650000 QueuedCount 4938 2016-04-07T04:13:46.5-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33097884, 26548942, 0 committed:false 2016-04-07T04:13:46.875-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33097884, 26548942, 0 committed:false) SnapCreateDur 10.332µs SnapOpenDur 374.911059ms 2016-04-07T04:13:47.301-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26660000 QueuedCount 5289 2016-04-07T04:13:47.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 837 2016-04-07T04:13:47.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 837 2016-04-07T04:13:47.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 837 2016-04-07T04:13:47.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 837 2016-04-07T04:13:47.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 837 2016-04-07T04:13:47.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 837 2016-04-07T04:13:47.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 837 2016-04-07T04:13:47.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 837 2016-04-07T04:13:47.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 837 2016-04-07T04:13:47.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 837 2016-04-07T04:13:47.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 837 2016-04-07T04:13:47.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 837 2016-04-07T04:13:47.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 837 2016-04-07T04:13:47.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 837 2016-04-07T04:13:47.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 837 2016-04-07T04:13:48.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 837 2016-04-07T04:13:48.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 837 2016-04-07T04:13:48.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 837 2016-04-07T04:13:48.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 837 2016-04-07T04:13:48.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 837 2016-04-07T04:13:48.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 837 2016-04-07T04:13:48.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 837 2016-04-07T04:13:48.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 837 2016-04-07T04:13:48.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 837 2016-04-07T04:13:48.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 837 2016-04-07T04:13:48.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 837 2016-04-07T04:13:48.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 837 2016-04-07T04:13:48.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 837 2016-04-07T04:13:48.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 837 2016-04-07T04:13:48.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 837 2016-04-07T04:13:48.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 837 2016-04-07T04:13:48.482-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.817µs CommitTime 721.449697ms TotalFlushTime 23m28.33659397s TotalCommitTime 2m18.957183441s 2016-04-07T04:13:48.482-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33118232, 26559116, 267 committed:true 2016-04-07T04:13:48.482-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33118232, 26559116, 267 committed:true) SnapCreateDur 731.76435ms SnapOpenDur 37.121µs 2016-04-07T04:13:48.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 837 2016-04-07T04:13:48.508-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:48.533-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:48.556-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:48.58-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:48.585-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35780000 2016-04-07T04:13:48.605-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:48.629-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35790000 2016-04-07T04:13:48.672-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35800000 2016-04-07T04:13:48.719-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35810000 2016-04-07T04:13:48.916-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26670000 QueuedCount 50000 2016-04-07T04:13:49.749-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26680000 QueuedCount 42807 2016-04-07T04:13:50.524-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26690000 QueuedCount 32807 2016-04-07T04:13:51.292-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26700000 QueuedCount 22807 2016-04-07T04:13:53.027-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26710000 QueuedCount 12807 2016-04-07T04:13:53.852-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26720000 QueuedCount 2807 2016-04-07T04:13:54.071-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 870 2016-04-07T04:13:54.084-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33231392, 26615696, 0 committed:false 2016-04-07T04:13:54.101-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 870 2016-04-07T04:13:54.131-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 870 2016-04-07T04:13:54.161-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 870 2016-04-07T04:13:54.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 870 2016-04-07T04:13:54.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 870 2016-04-07T04:13:54.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 870 2016-04-07T04:13:54.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 870 2016-04-07T04:13:54.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 870 2016-04-07T04:13:54.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 870 2016-04-07T04:13:54.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 870 2016-04-07T04:13:54.401-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 870 2016-04-07T04:13:54.431-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 870 2016-04-07T04:13:54.461-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 870 2016-04-07T04:13:54.491-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 870 2016-04-07T04:13:54.493-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33231392, 26615696, 0 committed:false) SnapCreateDur 13.449µs SnapOpenDur 409.188763ms 2016-04-07T04:13:55.058-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26730000 QueuedCount 3811 2016-04-07T04:13:56.777-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.344µs CommitTime 1.405752271s TotalFlushTime 23m34.703923132s TotalCommitTime 2m20.362935712s 2016-04-07T04:13:56.778-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33253058, 26626529, 268 committed:true 2016-04-07T04:13:56.778-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33253058, 26626529, 268 committed:true) SnapCreateDur 1.419062225s SnapOpenDur 30.373µs 2016-04-07T04:13:56.803-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:56.809-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35820000 2016-04-07T04:13:56.827-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:56.848-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35830000 2016-04-07T04:13:56.852-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:56.876-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:56.891-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35840000 2016-04-07T04:13:56.901-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:13:56.931-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35850000 2016-04-07T04:13:56.972-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35860000 2016-04-07T04:13:57.018-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35870000 2016-04-07T04:13:57.064-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35880000 2016-04-07T04:13:57.122-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35890000 2016-04-07T04:13:57.183-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35900000 2016-04-07T04:13:57.257-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35910000 2016-04-07T04:13:57.339-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35920000 2016-04-07T04:13:57.425-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35930000 2016-04-07T04:13:57.509-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35940000 2016-04-07T04:14:00.362-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26740000 QueuedCount 50000 2016-04-07T04:14:01.769-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26750000 QueuedCount 44671 2016-04-07T04:14:02.557-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26760000 QueuedCount 34671 2016-04-07T04:14:03.328-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26770000 QueuedCount 24671 2016-04-07T04:14:04.178-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26780000 QueuedCount 14671 2016-04-07T04:14:04.205-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 870 2016-04-07T04:14:04.235-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 870 2016-04-07T04:14:04.265-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 870 2016-04-07T04:14:04.295-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 870 2016-04-07T04:14:04.325-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 870 2016-04-07T04:14:04.355-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 870 2016-04-07T04:14:04.385-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 870 2016-04-07T04:14:04.415-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 870 2016-04-07T04:14:04.445-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 870 2016-04-07T04:14:04.475-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 870 2016-04-07T04:14:04.505-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 870 2016-04-07T04:14:04.535-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 870 2016-04-07T04:14:04.565-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 870 2016-04-07T04:14:04.595-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 870 2016-04-07T04:14:04.625-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 870 2016-04-07T04:14:04.655-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 870 2016-04-07T04:14:04.685-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 870 2016-04-07T04:14:04.715-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 870 2016-04-07T04:14:04.745-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 870 2016-04-07T04:14:04.775-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 870 2016-04-07T04:14:04.805-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 870 2016-04-07T04:14:04.835-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 870 2016-04-07T04:14:04.865-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 870 2016-04-07T04:14:04.895-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 870 2016-04-07T04:14:04.925-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 870 2016-04-07T04:14:04.955-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 870 2016-04-07T04:14:04.985-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 870 2016-04-07T04:14:05.009-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26790000 QueuedCount 4671 2016-04-07T04:14:05.015-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 870 2016-04-07T04:14:05.045-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 870 2016-04-07T04:14:05.075-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 870 2016-04-07T04:14:05.105-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 870 2016-04-07T04:14:05.135-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 870 2016-04-07T04:14:05.165-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 870 2016-04-07T04:14:05.195-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 870 2016-04-07T04:14:05.225-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 870 2016-04-07T04:14:05.255-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 870 2016-04-07T04:14:05.285-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 870 2016-04-07T04:14:05.315-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 870 2016-04-07T04:14:05.34-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33372832, 26686416, 0 committed:false 2016-04-07T04:14:05.345-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 870 2016-04-07T04:14:05.375-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 870 2016-04-07T04:14:05.405-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 870 2016-04-07T04:14:05.435-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 870 2016-04-07T04:14:05.465-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 870 2016-04-07T04:14:05.495-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 870 2016-04-07T04:14:05.525-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 870 2016-04-07T04:14:05.555-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 870 2016-04-07T04:14:05.585-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 870 2016-04-07T04:14:05.615-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 870 2016-04-07T04:14:05.645-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 870 2016-04-07T04:14:05.675-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 870 2016-04-07T04:14:05.705-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 870 2016-04-07T04:14:05.735-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 870 2016-04-07T04:14:05.752-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33372832, 26686416, 0 committed:false) SnapCreateDur 13.49µs SnapOpenDur 411.992809ms 2016-04-07T04:14:05.765-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 870 2016-04-07T04:14:05.795-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 870 2016-04-07T04:14:05.825-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 870 2016-04-07T04:14:05.855-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 870 2016-04-07T04:14:05.885-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 870 2016-04-07T04:14:05.915-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 870 2016-04-07T04:14:05.945-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 870 2016-04-07T04:14:05.975-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 870 2016-04-07T04:14:06.005-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 870 2016-04-07T04:14:06.035-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 870 2016-04-07T04:14:06.065-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 870 2016-04-07T04:14:06.095-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 870 2016-04-07T04:14:06.125-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 870 2016-04-07T04:14:06.155-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 870 2016-04-07T04:14:06.185-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 870 2016-04-07T04:14:06.215-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 870 2016-04-07T04:14:06.219-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26800000 QueuedCount 1670 2016-04-07T04:14:06.245-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 870 2016-04-07T04:14:06.275-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 870 2016-04-07T04:14:06.305-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 870 2016-04-07T04:14:06.335-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 870 2016-04-07T04:14:06.365-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 870 2016-04-07T04:14:06.395-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 870 2016-04-07T04:14:06.425-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 870 2016-04-07T04:14:06.455-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 870 2016-04-07T04:14:06.485-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 870 2016-04-07T04:14:06.515-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 870 2016-04-07T04:14:06.545-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 870 2016-04-07T04:14:06.575-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 870 2016-04-07T04:14:06.605-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 870 2016-04-07T04:14:06.635-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 870 2016-04-07T04:14:06.665-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 870 2016-04-07T04:14:06.695-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 870 2016-04-07T04:14:06.725-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 870 2016-04-07T04:14:06.755-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 870 2016-04-07T04:14:06.785-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 870 2016-04-07T04:14:06.815-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 870 2016-04-07T04:14:06.845-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 870 2016-04-07T04:14:06.875-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 870 2016-04-07T04:14:06.905-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 870 2016-04-07T04:14:06.935-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 870 2016-04-07T04:14:06.965-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 870 2016-04-07T04:14:06.995-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 870 2016-04-07T04:14:07.025-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 870 2016-04-07T04:14:07.055-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 870 2016-04-07T04:14:07.085-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 870 2016-04-07T04:14:07.115-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 870 2016-04-07T04:14:07.145-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 870 2016-04-07T04:14:07.175-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 870 2016-04-07T04:14:07.205-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 870 2016-04-07T04:14:07.235-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 870 2016-04-07T04:14:07.265-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 870 2016-04-07T04:14:07.295-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 870 2016-04-07T04:14:07.325-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 870 2016-04-07T04:14:07.355-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 870 2016-04-07T04:14:07.385-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 870 2016-04-07T04:14:07.386-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.507µs CommitTime 1.039426193s TotalFlushTime 23m43.746135741s TotalCommitTime 2m21.402361905s 2016-04-07T04:14:07.386-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33386634, 26693317, 269 committed:true 2016-04-07T04:14:07.386-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33386634, 26693317, 269 committed:true) SnapCreateDur 1.050717939s SnapOpenDur 39.241µs 2016-04-07T04:14:07.409-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:07.433-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:07.457-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:07.481-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:07.505-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:08.114-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26810000 QueuedCount 38368 2016-04-07T04:14:09.044-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26820000 QueuedCount 28368 2016-04-07T04:14:09.836-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26830000 QueuedCount 18368 2016-04-07T04:14:10.59-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26840000 QueuedCount 8368 2016-04-07T04:14:11.323-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33478514, 26739257, 0 committed:false 2016-04-07T04:14:11.685-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33478514, 26739257, 0 committed:false) SnapCreateDur 15.307µs SnapOpenDur 361.445559ms 2016-04-07T04:14:11.824-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26850000 QueuedCount 8552 2016-04-07T04:14:12.949-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":9090132,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":4090910921,"bucket-1:myindex:avg_ts_items_count":35942,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":7816168615,"bucket-1:myindex:delete_bytes":289991355,"bucket-1:myindex:disk_size":65369939968,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":88,"bucket-1:myindex:get_bytes":294686396,"bucket-1:myindex:insert_bytes":2808118320,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":269,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":27282396,"bucket-1:myindex:num_docs_pending":2149127,"bucket-1:myindex:num_docs_queued":10568477,"bucket-1:myindex:num_flush_queued":26858552,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":388,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":4298342069,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"269 141402361905 -6189464753380886077","bucket-1:myindex:timings/storage_del":"6749283 174666900925 -5446544039016496619","bucket-1:myindex:timings/storage_get":"26858552 193904127001 662859359478058951","bucket-1:myindex:timings/storage_handle_open":"269 6440696 161674149086","bucket-1:myindex:timings/storage_info":"1318 11565096 282349172064","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"273 187499370 3681949749970126","bucket-1:myindex:timings/storage_meta_set":"270 440512563 107863487039016117","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"53498566 1042350459474 8191127454532957112","bucket-1:myindex:timings/storage_snapshot_close":"118 4675039885 196149212827300377","bucket-1:myindex:timings/storage_snapshot_create":"119 44038986690 7057322955248423492","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":35948684,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":785,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"5007 755192760 134467396975010"} 2016-04-07T04:14:13.396-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 12.754µs CommitTime 915.627842ms TotalFlushTime 23m48.396095389s TotalCommitTime 2m22.317989747s 2016-04-07T04:14:13.396-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33498566, 26749283, 270 committed:true 2016-04-07T04:14:13.396-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33498566, 26749283, 270 committed:true) SnapCreateDur 926.142765ms SnapOpenDur 53.018µs 2016-04-07T04:14:13.423-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:13.428-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35950000 2016-04-07T04:14:13.45-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:13.469-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35960000 2016-04-07T04:14:13.475-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:13.499-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:13.507-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35970000 2016-04-07T04:14:13.545-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35980000 2016-04-07T04:14:13.553-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26860000 QueuedCount 50000 2016-04-07T04:14:13.578-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 35990000 2016-04-07T04:14:13.619-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36000000 2016-04-07T04:14:13.659-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36010000 2016-04-07T04:14:13.696-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36020000 2016-04-07T04:14:13.736-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36030000 2016-04-07T04:14:13.779-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36040000 2016-04-07T04:14:13.821-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36050000 2016-04-07T04:14:13.821-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:13.869-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36060000 2016-04-07T04:14:14.472-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26870000 QueuedCount 50000 2016-04-07T04:14:15.304-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26880000 QueuedCount 41252 2016-04-07T04:14:17.251-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26890000 QueuedCount 31252 2016-04-07T04:14:19.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 870 2016-04-07T04:14:19.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 870 2016-04-07T04:14:19.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 870 2016-04-07T04:14:19.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 870 2016-04-07T04:14:19.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 870 2016-04-07T04:14:19.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 870 2016-04-07T04:14:19.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 870 2016-04-07T04:14:19.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 870 2016-04-07T04:14:19.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 870 2016-04-07T04:14:19.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 870 2016-04-07T04:14:19.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 870 2016-04-07T04:14:19.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 870 2016-04-07T04:14:19.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 870 2016-04-07T04:14:19.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 870 2016-04-07T04:14:19.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 870 2016-04-07T04:14:19.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 870 2016-04-07T04:14:19.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 870 2016-04-07T04:14:19.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 870 2016-04-07T04:14:19.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 870 2016-04-07T04:14:19.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 870 2016-04-07T04:14:19.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 870 2016-04-07T04:14:20.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 870 2016-04-07T04:14:20.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 870 2016-04-07T04:14:20.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 870 2016-04-07T04:14:20.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 870 2016-04-07T04:14:20.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 870 2016-04-07T04:14:20.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 870 2016-04-07T04:14:20.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 870 2016-04-07T04:14:20.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 870 2016-04-07T04:14:20.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 870 2016-04-07T04:14:20.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 870 2016-04-07T04:14:20.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 870 2016-04-07T04:14:20.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 870 2016-04-07T04:14:20.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 870 2016-04-07T04:14:20.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 870 2016-04-07T04:14:20.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 870 2016-04-07T04:14:20.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 870 2016-04-07T04:14:20.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 870 2016-04-07T04:14:20.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 870 2016-04-07T04:14:20.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 870 2016-04-07T04:14:20.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 870 2016-04-07T04:14:20.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 870 2016-04-07T04:14:20.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 870 2016-04-07T04:14:20.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 870 2016-04-07T04:14:20.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 870 2016-04-07T04:14:20.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 870 2016-04-07T04:14:20.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 870 2016-04-07T04:14:20.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 870 2016-04-07T04:14:20.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 870 2016-04-07T04:14:20.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 870 2016-04-07T04:14:20.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 870 2016-04-07T04:14:20.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 870 2016-04-07T04:14:20.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 870 2016-04-07T04:14:20.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 870 2016-04-07T04:14:21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 870 2016-04-07T04:14:21.03-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 870 2016-04-07T04:14:21.06-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 870 2016-04-07T04:14:21.09-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 870 2016-04-07T04:14:21.12-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 870 2016-04-07T04:14:21.15-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 870 2016-04-07T04:14:21.18-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 870 2016-04-07T04:14:21.21-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 870 2016-04-07T04:14:21.24-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 870 2016-04-07T04:14:21.27-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 870 2016-04-07T04:14:21.3-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 870 2016-04-07T04:14:21.33-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 870 2016-04-07T04:14:21.36-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 870 2016-04-07T04:14:21.39-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 870 2016-04-07T04:14:21.42-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 870 2016-04-07T04:14:21.45-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 870 2016-04-07T04:14:21.48-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 870 2016-04-07T04:14:21.51-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 870 2016-04-07T04:14:21.54-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 870 2016-04-07T04:14:21.57-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 870 2016-04-07T04:14:21.6-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 870 2016-04-07T04:14:21.63-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 870 2016-04-07T04:14:21.66-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 870 2016-04-07T04:14:21.69-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 870 2016-04-07T04:14:21.72-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 870 2016-04-07T04:14:21.75-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 870 2016-04-07T04:14:21.78-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 870 2016-04-07T04:14:21.81-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 870 2016-04-07T04:14:21.84-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 870 2016-04-07T04:14:21.87-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 870 2016-04-07T04:14:21.889-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26900000 QueuedCount 21252 2016-04-07T04:14:21.9-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 870 2016-04-07T04:14:21.93-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 870 2016-04-07T04:14:21.96-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 870 2016-04-07T04:14:21.99-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 870 2016-04-07T04:14:22.02-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 870 2016-04-07T04:14:22.05-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 870 2016-04-07T04:14:22.08-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 870 2016-04-07T04:14:22.11-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 870 2016-04-07T04:14:22.14-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 870 2016-04-07T04:14:22.17-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 870 2016-04-07T04:14:22.2-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 870 2016-04-07T04:14:22.23-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 870 2016-04-07T04:14:22.26-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 870 2016-04-07T04:14:22.29-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 870 2016-04-07T04:14:22.32-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 870 2016-04-07T04:14:22.35-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 870 2016-04-07T04:14:22.38-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 870 2016-04-07T04:14:22.41-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 870 2016-04-07T04:14:22.44-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 870 2016-04-07T04:14:22.47-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 870 2016-04-07T04:14:22.5-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 870 2016-04-07T04:14:22.53-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 870 2016-04-07T04:14:22.56-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 870 2016-04-07T04:14:22.59-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 870 2016-04-07T04:14:22.62-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 870 2016-04-07T04:14:22.65-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 870 2016-04-07T04:14:22.68-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 870 2016-04-07T04:14:22.71-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 870 2016-04-07T04:14:22.729-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26910000 QueuedCount 11252 2016-04-07T04:14:22.74-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 870 2016-04-07T04:14:22.77-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 870 2016-04-07T04:14:22.8-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 870 2016-04-07T04:14:22.83-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 870 2016-04-07T04:14:22.86-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 870 2016-04-07T04:14:22.89-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 870 2016-04-07T04:14:22.92-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 870 2016-04-07T04:14:22.95-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 870 2016-04-07T04:14:22.98-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 870 2016-04-07T04:14:23.01-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 870 2016-04-07T04:14:23.04-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 870 2016-04-07T04:14:23.07-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8700 Milliseconds Vbucket 870 2016-04-07T04:14:23.1-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8730 Milliseconds Vbucket 870 2016-04-07T04:14:23.13-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8760 Milliseconds Vbucket 870 2016-04-07T04:14:23.16-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8790 Milliseconds Vbucket 870 2016-04-07T04:14:23.19-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8820 Milliseconds Vbucket 870 2016-04-07T04:14:23.22-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8850 Milliseconds Vbucket 870 2016-04-07T04:14:23.25-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8880 Milliseconds Vbucket 870 2016-04-07T04:14:23.28-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8910 Milliseconds Vbucket 870 2016-04-07T04:14:23.31-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8940 Milliseconds Vbucket 870 2016-04-07T04:14:23.34-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8970 Milliseconds Vbucket 870 2016-04-07T04:14:23.37-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9000 Milliseconds Vbucket 870 2016-04-07T04:14:23.4-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9030 Milliseconds Vbucket 870 2016-04-07T04:14:23.43-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9060 Milliseconds Vbucket 870 2016-04-07T04:14:23.46-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9090 Milliseconds Vbucket 870 2016-04-07T04:14:23.49-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9120 Milliseconds Vbucket 870 2016-04-07T04:14:23.52-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9150 Milliseconds Vbucket 870 2016-04-07T04:14:23.537-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26920000 QueuedCount 1252 2016-04-07T04:14:23.55-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9180 Milliseconds Vbucket 870 2016-04-07T04:14:23.566-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33622010, 26811005, 0 committed:false 2016-04-07T04:14:23.58-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9210 Milliseconds Vbucket 870 2016-04-07T04:14:23.61-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9240 Milliseconds Vbucket 870 2016-04-07T04:14:23.64-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9270 Milliseconds Vbucket 870 2016-04-07T04:14:23.67-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9300 Milliseconds Vbucket 870 2016-04-07T04:14:23.7-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9330 Milliseconds Vbucket 870 2016-04-07T04:14:23.73-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9360 Milliseconds Vbucket 870 2016-04-07T04:14:23.76-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9390 Milliseconds Vbucket 870 2016-04-07T04:14:23.79-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9420 Milliseconds Vbucket 870 2016-04-07T04:14:23.82-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9450 Milliseconds Vbucket 870 2016-04-07T04:14:23.85-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9480 Milliseconds Vbucket 870 2016-04-07T04:14:23.88-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9510 Milliseconds Vbucket 870 2016-04-07T04:14:23.91-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9540 Milliseconds Vbucket 870 2016-04-07T04:14:23.94-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9570 Milliseconds Vbucket 870 2016-04-07T04:14:23.97-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9600 Milliseconds Vbucket 870 2016-04-07T04:14:23.972-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33622010, 26811005, 0 committed:false) SnapCreateDur 11.07µs SnapOpenDur 405.296994ms 2016-04-07T04:14:25.56-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.714µs CommitTime 883.232011ms TotalFlushTime 23m59.146733671s TotalCommitTime 2m23.201221758s 2016-04-07T04:14:25.56-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33638710, 26819355, 271 committed:true 2016-04-07T04:14:25.56-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33638710, 26819355, 271 committed:true) SnapCreateDur 893.949905ms SnapOpenDur 49.321µs 2016-04-07T04:14:25.584-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:25.587-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26930000 QueuedCount 6779 2016-04-07T04:14:25.592-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36070000 2016-04-07T04:14:25.608-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:25.629-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36080000 2016-04-07T04:14:25.633-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:25.657-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:25.681-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:25.705-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36090000 2016-04-07T04:14:25.752-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36100000 2016-04-07T04:14:25.795-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36110000 2016-04-07T04:14:26.505-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26940000 QueuedCount 45147 2016-04-07T04:14:27.31-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26950000 QueuedCount 35147 2016-04-07T04:14:28.152-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26960000 QueuedCount 25147 2016-04-07T04:14:28.996-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26970000 QueuedCount 15147 2016-04-07T04:14:31.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 870 2016-04-07T04:14:31.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 870 2016-04-07T04:14:31.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 870 2016-04-07T04:14:31.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 870 2016-04-07T04:14:31.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 870 2016-04-07T04:14:31.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 870 2016-04-07T04:14:31.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 870 2016-04-07T04:14:31.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 870 2016-04-07T04:14:31.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 870 2016-04-07T04:14:31.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 870 2016-04-07T04:14:31.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 870 2016-04-07T04:14:31.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 870 2016-04-07T04:14:31.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 870 2016-04-07T04:14:31.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 870 2016-04-07T04:14:31.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 870 2016-04-07T04:14:31.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 870 2016-04-07T04:14:31.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 870 2016-04-07T04:14:31.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 870 2016-04-07T04:14:31.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 870 2016-04-07T04:14:31.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 870 2016-04-07T04:14:31.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 870 2016-04-07T04:14:31.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 870 2016-04-07T04:14:31.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 870 2016-04-07T04:14:31.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 870 2016-04-07T04:14:31.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 870 2016-04-07T04:14:31.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 870 2016-04-07T04:14:31.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 870 2016-04-07T04:14:31.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 870 2016-04-07T04:14:31.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 870 2016-04-07T04:14:31.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 870 2016-04-07T04:14:31.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 870 2016-04-07T04:14:31.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 870 2016-04-07T04:14:31.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 870 2016-04-07T04:14:32.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 870 2016-04-07T04:14:32.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 870 2016-04-07T04:14:32.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 870 2016-04-07T04:14:32.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 870 2016-04-07T04:14:32.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 870 2016-04-07T04:14:32.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 870 2016-04-07T04:14:32.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 870 2016-04-07T04:14:32.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 870 2016-04-07T04:14:32.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 870 2016-04-07T04:14:32.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 870 2016-04-07T04:14:32.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 870 2016-04-07T04:14:32.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 870 2016-04-07T04:14:32.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 870 2016-04-07T04:14:32.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 870 2016-04-07T04:14:32.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 870 2016-04-07T04:14:32.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 870 2016-04-07T04:14:32.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 870 2016-04-07T04:14:32.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 870 2016-04-07T04:14:32.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 870 2016-04-07T04:14:32.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 870 2016-04-07T04:14:32.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 870 2016-04-07T04:14:32.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 870 2016-04-07T04:14:32.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 870 2016-04-07T04:14:32.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 870 2016-04-07T04:14:32.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 870 2016-04-07T04:14:32.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 870 2016-04-07T04:14:32.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 870 2016-04-07T04:14:32.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 870 2016-04-07T04:14:32.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 870 2016-04-07T04:14:32.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 870 2016-04-07T04:14:32.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 870 2016-04-07T04:14:32.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 870 2016-04-07T04:14:32.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 870 2016-04-07T04:14:33.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 870 2016-04-07T04:14:33.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 870 2016-04-07T04:14:33.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 870 2016-04-07T04:14:33.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 870 2016-04-07T04:14:33.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 870 2016-04-07T04:14:33.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 870 2016-04-07T04:14:33.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 870 2016-04-07T04:14:33.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 870 2016-04-07T04:14:33.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 870 2016-04-07T04:14:33.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 870 2016-04-07T04:14:33.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 870 2016-04-07T04:14:33.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 870 2016-04-07T04:14:33.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 870 2016-04-07T04:14:33.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 870 2016-04-07T04:14:33.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 870 2016-04-07T04:14:33.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 870 2016-04-07T04:14:33.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 870 2016-04-07T04:14:33.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 870 2016-04-07T04:14:33.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 870 2016-04-07T04:14:33.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 870 2016-04-07T04:14:33.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 870 2016-04-07T04:14:33.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 870 2016-04-07T04:14:33.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 870 2016-04-07T04:14:33.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 870 2016-04-07T04:14:33.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 870 2016-04-07T04:14:33.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 870 2016-04-07T04:14:33.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 870 2016-04-07T04:14:33.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 870 2016-04-07T04:14:33.844-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26980000 QueuedCount 5147 2016-04-07T04:14:33.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 870 2016-04-07T04:14:33.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 870 2016-04-07T04:14:33.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 870 2016-04-07T04:14:33.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 870 2016-04-07T04:14:33.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 870 2016-04-07T04:14:34.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 870 2016-04-07T04:14:34.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 870 2016-04-07T04:14:34.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 870 2016-04-07T04:14:34.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 870 2016-04-07T04:14:34.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 870 2016-04-07T04:14:34.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 870 2016-04-07T04:14:34.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 870 2016-04-07T04:14:34.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 870 2016-04-07T04:14:34.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 870 2016-04-07T04:14:34.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 870 2016-04-07T04:14:34.285-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33747748, 26873874, 0 committed:false 2016-04-07T04:14:34.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 870 2016-04-07T04:14:34.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 870 2016-04-07T04:14:34.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 870 2016-04-07T04:14:34.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 870 2016-04-07T04:14:34.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 870 2016-04-07T04:14:34.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 870 2016-04-07T04:14:34.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 870 2016-04-07T04:14:34.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 870 2016-04-07T04:14:34.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 870 2016-04-07T04:14:34.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 870 2016-04-07T04:14:34.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 870 2016-04-07T04:14:34.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 870 2016-04-07T04:14:34.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 870 2016-04-07T04:14:34.693-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33747748, 26873874, 0 committed:false) SnapCreateDur 21.276µs SnapOpenDur 408.084093ms 2016-04-07T04:14:34.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 870 2016-04-07T04:14:34.73-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36120000 2016-04-07T04:14:34.782-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36130000 2016-04-07T04:14:35.16-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 26990000 QueuedCount 4429 2016-04-07T04:14:36.357-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.949µs CommitTime 816.415902ms TotalFlushTime 24m8.613056247s TotalCommitTime 2m24.01763766s 2016-04-07T04:14:36.357-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33766038, 26883019, 272 committed:true 2016-04-07T04:14:36.357-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33766038, 26883019, 272 committed:true) SnapCreateDur 829.191577ms SnapOpenDur 32.357µs 2016-04-07T04:14:36.382-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:36.405-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:36.427-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:36.45-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:36.474-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:36.795-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27000000 QueuedCount 36927 2016-04-07T04:14:37.718-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27010000 QueuedCount 26927 2016-04-07T04:14:38.482-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27020000 QueuedCount 16927 2016-04-07T04:14:39.241-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27030000 QueuedCount 6927 2016-04-07T04:14:39.799-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33849696, 26924848, 0 committed:false 2016-04-07T04:14:40.139-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33849696, 26924848, 0 committed:false) SnapCreateDur 17.743µs SnapOpenDur 340.034021ms 2016-04-07T04:14:40.395-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27040000 QueuedCount 8341 2016-04-07T04:14:41.849-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.368µs CommitTime 792.400484ms TotalFlushTime 24m12.88737777s TotalCommitTime 2m24.810038144s 2016-04-07T04:14:41.849-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33872162, 26936081, 273 committed:true 2016-04-07T04:14:41.849-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33872162, 26936081, 273 committed:true) SnapCreateDur 803.680088ms SnapOpenDur 94.534µs 2016-04-07T04:14:41.874-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:41.9-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:41.918-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36140000 2016-04-07T04:14:41.926-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:41.951-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:41.958-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36150000 2016-04-07T04:14:41.998-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36160000 2016-04-07T04:14:42.03-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27050000 QueuedCount 50000 2016-04-07T04:14:42.036-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36170000 2016-04-07T04:14:42.071-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36180000 2016-04-07T04:14:42.112-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36190000 2016-04-07T04:14:42.154-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36200000 2016-04-07T04:14:42.176-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:42.199-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36210000 2016-04-07T04:14:43.014-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27060000 QueuedCount 50000 2016-04-07T04:14:43.059-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36220000 2016-04-07T04:14:43.106-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36230000 2016-04-07T04:14:43.157-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36240000 2016-04-07T04:14:44.598-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27070000 QueuedCount 41095 2016-04-07T04:14:45.369-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27080000 QueuedCount 31095 2016-04-07T04:14:46.171-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27090000 QueuedCount 21095 2016-04-07T04:14:47.024-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27100000 QueuedCount 11095 2016-04-07T04:14:47.776-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27110000 QueuedCount 1095 2016-04-07T04:14:47.799-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 33995688, 26997844, 0 committed:false 2016-04-07T04:14:48.191-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 837 2016-04-07T04:14:48.221-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 837 2016-04-07T04:14:48.251-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 837 2016-04-07T04:14:48.281-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 837 2016-04-07T04:14:48.311-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 837 2016-04-07T04:14:48.341-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 837 2016-04-07T04:14:48.356-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 33995688, 26997844, 0 committed:false) SnapCreateDur 10.715µs SnapOpenDur 557.533274ms 2016-04-07T04:14:48.371-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 837 2016-04-07T04:14:49.257-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27120000 QueuedCount 333 2016-04-07T04:14:50.131-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.796µs CommitTime 848.882817ms TotalFlushTime 24m19.644000247s TotalCommitTime 2m25.658920961s 2016-04-07T04:14:50.131-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34013862, 27006931, 274 committed:true 2016-04-07T04:14:50.131-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34013862, 27006931, 274 committed:true) SnapCreateDur 859.428719ms SnapOpenDur 54.845µs 2016-04-07T04:14:50.155-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:50.179-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:50.202-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:50.227-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:50.25-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:14:50.281-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36250000 2016-04-07T04:14:50.319-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36260000 2016-04-07T04:14:50.356-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36270000 2016-04-07T04:14:50.398-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36280000 2016-04-07T04:14:50.445-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36290000 2016-04-07T04:14:50.489-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36300000 2016-04-07T04:14:50.543-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36310000 2016-04-07T04:14:50.595-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36320000 2016-04-07T04:14:50.649-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36330000 2016-04-07T04:14:51.008-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27130000 QueuedCount 50000 2016-04-07T04:14:51.946-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27140000 QueuedCount 43485 2016-04-07T04:14:52.783-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27150000 QueuedCount 33485 2016-04-07T04:14:56.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 870 2016-04-07T04:14:56.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 870 2016-04-07T04:14:56.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 870 2016-04-07T04:14:56.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 870 2016-04-07T04:14:56.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 870 2016-04-07T04:14:56.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 870 2016-04-07T04:14:56.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 870 2016-04-07T04:14:56.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 870 2016-04-07T04:14:56.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 870 2016-04-07T04:14:56.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 870 2016-04-07T04:14:56.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 870 2016-04-07T04:14:56.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 870 2016-04-07T04:14:56.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 870 2016-04-07T04:14:56.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 870 2016-04-07T04:14:56.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 870 2016-04-07T04:14:56.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 870 2016-04-07T04:14:56.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 870 2016-04-07T04:14:56.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 870 2016-04-07T04:14:56.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 870 2016-04-07T04:14:56.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 870 2016-04-07T04:14:56.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 870 2016-04-07T04:14:56.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 870 2016-04-07T04:14:56.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 870 2016-04-07T04:14:56.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 870 2016-04-07T04:14:56.845-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27160000 QueuedCount 23485 2016-04-07T04:14:56.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 870 2016-04-07T04:14:56.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 870 2016-04-07T04:14:56.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 870 2016-04-07T04:14:56.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 870 2016-04-07T04:14:56.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 870 2016-04-07T04:14:57.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 870 2016-04-07T04:14:57.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 870 2016-04-07T04:14:57.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 870 2016-04-07T04:14:57.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 870 2016-04-07T04:14:57.127-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 870 2016-04-07T04:14:57.157-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 870 2016-04-07T04:14:57.187-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 870 2016-04-07T04:14:57.217-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 870 2016-04-07T04:14:57.247-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 870 2016-04-07T04:14:57.277-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 870 2016-04-07T04:14:57.307-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 870 2016-04-07T04:14:57.337-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 870 2016-04-07T04:14:57.367-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 870 2016-04-07T04:14:57.397-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 870 2016-04-07T04:14:57.427-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 870 2016-04-07T04:14:57.457-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 870 2016-04-07T04:14:57.487-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 870 2016-04-07T04:14:57.517-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 870 2016-04-07T04:14:57.547-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 870 2016-04-07T04:14:57.577-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 870 2016-04-07T04:14:57.607-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 870 2016-04-07T04:14:57.637-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 870 2016-04-07T04:14:57.667-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 870 2016-04-07T04:14:57.697-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 870 2016-04-07T04:14:57.727-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 870 2016-04-07T04:14:57.757-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 870 2016-04-07T04:14:57.787-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 870 2016-04-07T04:14:57.817-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 870 2016-04-07T04:14:57.847-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 870 2016-04-07T04:14:57.877-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 870 2016-04-07T04:14:57.907-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 870 2016-04-07T04:14:57.937-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 870 2016-04-07T04:14:57.967-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 870 2016-04-07T04:14:57.997-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 870 2016-04-07T04:14:58.027-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 870 2016-04-07T04:14:58.057-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 870 2016-04-07T04:14:58.087-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 870 2016-04-07T04:14:58.117-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 870 2016-04-07T04:14:58.147-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 870 2016-04-07T04:14:58.177-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 870 2016-04-07T04:14:58.207-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 870 2016-04-07T04:14:58.237-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 870 2016-04-07T04:14:58.267-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 870 2016-04-07T04:14:58.297-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 870 2016-04-07T04:14:58.327-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 870 2016-04-07T04:14:58.357-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 870 2016-04-07T04:14:58.387-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 870 2016-04-07T04:14:58.417-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 870 2016-04-07T04:14:58.423-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27170000 QueuedCount 13485 2016-04-07T04:14:58.447-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 870 2016-04-07T04:14:58.477-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 870 2016-04-07T04:14:58.507-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 870 2016-04-07T04:14:58.537-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 870 2016-04-07T04:14:58.567-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 870 2016-04-07T04:14:58.597-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 870 2016-04-07T04:14:58.627-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 870 2016-04-07T04:14:58.657-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 870 2016-04-07T04:14:58.687-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 870 2016-04-07T04:14:58.717-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 870 2016-04-07T04:14:58.747-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 870 2016-04-07T04:14:58.777-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 870 2016-04-07T04:14:58.807-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 870 2016-04-07T04:14:58.837-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 870 2016-04-07T04:14:58.867-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 870 2016-04-07T04:14:58.897-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 870 2016-04-07T04:14:58.927-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 870 2016-04-07T04:14:58.957-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 870 2016-04-07T04:14:58.987-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 870 2016-04-07T04:14:59.017-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 870 2016-04-07T04:14:59.047-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 870 2016-04-07T04:14:59.077-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 870 2016-04-07T04:14:59.107-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 870 2016-04-07T04:14:59.137-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 870 2016-04-07T04:14:59.167-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 870 2016-04-07T04:14:59.197-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 870 2016-04-07T04:14:59.227-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 870 2016-04-07T04:14:59.257-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 870 2016-04-07T04:14:59.287-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 870 2016-04-07T04:14:59.299-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27180000 QueuedCount 3485 2016-04-07T04:14:59.317-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 870 2016-04-07T04:14:59.347-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 870 2016-04-07T04:14:59.377-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 870 2016-04-07T04:14:59.407-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 870 2016-04-07T04:14:59.437-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 870 2016-04-07T04:14:59.467-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 870 2016-04-07T04:14:59.497-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 870 2016-04-07T04:14:59.527-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 870 2016-04-07T04:14:59.557-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 870 2016-04-07T04:14:59.587-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 870 2016-04-07T04:14:59.617-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 870 2016-04-07T04:14:59.647-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 870 2016-04-07T04:14:59.656-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34138196, 27069098, 0 committed:false 2016-04-07T04:14:59.677-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 870 2016-04-07T04:14:59.707-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 870 2016-04-07T04:14:59.737-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 870 2016-04-07T04:14:59.767-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 870 2016-04-07T04:14:59.797-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 870 2016-04-07T04:14:59.827-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8700 Milliseconds Vbucket 870 2016-04-07T04:14:59.857-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8730 Milliseconds Vbucket 870 2016-04-07T04:14:59.887-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8760 Milliseconds Vbucket 870 2016-04-07T04:14:59.917-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8790 Milliseconds Vbucket 870 2016-04-07T04:14:59.947-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8820 Milliseconds Vbucket 870 2016-04-07T04:14:59.977-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8850 Milliseconds Vbucket 870 2016-04-07T04:15:00.007-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8880 Milliseconds Vbucket 870 2016-04-07T04:15:00.037-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8910 Milliseconds Vbucket 870 2016-04-07T04:15:00.067-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8940 Milliseconds Vbucket 870 2016-04-07T04:15:00.091-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34138196, 27069098, 0 committed:false) SnapCreateDur 16.739µs SnapOpenDur 435.063648ms 2016-04-07T04:15:00.097-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8970 Milliseconds Vbucket 870 2016-04-07T04:15:00.15-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36340000 2016-04-07T04:15:00.63-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27190000 QueuedCount 4169 2016-04-07T04:15:01.936-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 11.85µs CommitTime 929.073952ms TotalFlushTime 24m29.963284871s TotalCommitTime 2m26.587994913s 2016-04-07T04:15:01.936-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34159258, 27079629, 275 committed:true 2016-04-07T04:15:01.936-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34159258, 27079629, 275 committed:true) SnapCreateDur 940.394684ms SnapOpenDur 46.909µs 2016-04-07T04:15:01.961-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:01.985-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:01.991-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36350000 2016-04-07T04:15:02.008-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:02.033-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:02.037-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36360000 2016-04-07T04:15:02.056-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:02.08-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36370000 2016-04-07T04:15:02.12-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36380000 2016-04-07T04:15:02.169-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36390000 2016-04-07T04:15:02.52-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27200000 QueuedCount 34594 2016-04-07T04:15:03.376-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27210000 QueuedCount 24594 2016-04-07T04:15:04.241-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27220000 QueuedCount 14594 2016-04-07T04:15:05.264-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27230000 QueuedCount 4594 2016-04-07T04:15:05.629-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34238872, 27119436, 0 committed:false 2016-04-07T04:15:05.946-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34238872, 27119436, 0 committed:false) SnapCreateDur 16.849µs SnapOpenDur 317.010436ms 2016-04-07T04:15:06.422-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27240000 QueuedCount 2455 2016-04-07T04:15:07.418-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.408µs CommitTime 747.134543ms TotalFlushTime 24m34.290606862s TotalCommitTime 2m27.335129456s 2016-04-07T04:15:07.418-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34254332, 27127166, 276 committed:true 2016-04-07T04:15:07.418-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34254332, 27127166, 276 committed:true) SnapCreateDur 758.13239ms SnapOpenDur 43.839µs 2016-04-07T04:15:07.443-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:07.469-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:07.475-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36400000 2016-04-07T04:15:07.493-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:07.513-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36410000 2016-04-07T04:15:07.517-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:12.482-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:12.734-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27250000 QueuedCount 50000 2016-04-07T04:15:12.95-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":9116444,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2793252896,"bucket-1:myindex:avg_ts_items_count":20940,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":7887706792,"bucket-1:myindex:delete_bytes":306645040,"bucket-1:myindex:disk_size":68917284864,"bucket-1:myindex:flush_queue_size":50842,"bucket-1:myindex:frag_percent":88,"bucket-1:myindex:get_bytes":311605434,"bucket-1:myindex:insert_bytes":2848784295,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":276,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":27732553,"bucket-1:myindex:num_docs_pending":690161,"bucket-1:myindex:num_docs_queued":11577286,"bucket-1:myindex:num_flush_queued":27303037,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":401,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1472429833,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"276 147335129456 -1134567700371724490","bucket-1:myindex:timings/storage_del":"7136754 192891754335 4052278739376789049","bucket-1:myindex:timings/storage_get":"27252196 199265923567 718265195009957347","bucket-1:myindex:timings/storage_handle_open":"276 6647759 170583058269","bucket-1:myindex:timings/storage_info":"1355 11894709 289367776317","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"279 189935711 3682964707477995","bucket-1:myindex:timings/storage_meta_set":"276 443448008 107864929361898676","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"54273508 1069833544260 -3570868944396655092","bucket-1:myindex:timings/storage_snapshot_close":"125 5001143648 211548025041994686","bucket-1:myindex:timings/storage_snapshot_create":"125 46501627606 8104057958133411596","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36418640,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":775,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"5149 781121134 139306975171906"} 2016-04-07T04:15:13.313-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36420000 2016-04-07T04:15:13.677-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27260000 QueuedCount 49008 2016-04-07T04:15:14.71-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27270000 QueuedCount 39008 2016-04-07T04:15:15.513-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27280000 QueuedCount 29008 2016-04-07T04:15:16.4-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27290000 QueuedCount 19008 2016-04-07T04:15:17.652-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27300000 QueuedCount 9008 2016-04-07T04:15:18.823-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 837 2016-04-07T04:15:18.853-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 837 2016-04-07T04:15:18.883-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 837 2016-04-07T04:15:18.913-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 837 2016-04-07T04:15:18.943-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 837 2016-04-07T04:15:18.973-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 837 2016-04-07T04:15:19.003-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 837 2016-04-07T04:15:19.033-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 837 2016-04-07T04:15:19.063-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 837 2016-04-07T04:15:19.093-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 837 2016-04-07T04:15:19.123-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 837 2016-04-07T04:15:19.153-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 837 2016-04-07T04:15:19.183-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 837 2016-04-07T04:15:19.213-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 837 2016-04-07T04:15:19.243-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 837 2016-04-07T04:15:19.273-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 837 2016-04-07T04:15:19.303-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 837 2016-04-07T04:15:19.333-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 837 2016-04-07T04:15:19.363-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 837 2016-04-07T04:15:19.393-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 837 2016-04-07T04:15:19.423-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 837 2016-04-07T04:15:19.453-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 837 2016-04-07T04:15:19.483-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 837 2016-04-07T04:15:19.513-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 837 2016-04-07T04:15:19.543-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 837 2016-04-07T04:15:19.573-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 837 2016-04-07T04:15:19.603-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 837 2016-04-07T04:15:19.633-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 837 2016-04-07T04:15:19.663-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 837 2016-04-07T04:15:19.693-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 837 2016-04-07T04:15:19.723-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 837 2016-04-07T04:15:19.753-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 837 2016-04-07T04:15:19.783-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 837 2016-04-07T04:15:19.813-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 837 2016-04-07T04:15:19.843-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 837 2016-04-07T04:15:19.873-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 837 2016-04-07T04:15:19.903-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 837 2016-04-07T04:15:19.933-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 837 2016-04-07T04:15:19.963-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 837 2016-04-07T04:15:19.993-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 837 2016-04-07T04:15:20.023-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 837 2016-04-07T04:15:20.053-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 837 2016-04-07T04:15:20.083-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 837 2016-04-07T04:15:20.113-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 837 2016-04-07T04:15:20.143-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 837 2016-04-07T04:15:20.173-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 837 2016-04-07T04:15:20.203-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 837 2016-04-07T04:15:20.233-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 837 2016-04-07T04:15:20.263-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 837 2016-04-07T04:15:20.293-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 837 2016-04-07T04:15:20.323-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 837 2016-04-07T04:15:20.353-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 837 2016-04-07T04:15:20.365-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.59µs CommitTime 827.107474ms TotalFlushTime 24m46.310115517s TotalCommitTime 2m28.16223693s 2016-04-07T04:15:20.365-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34385322, 27192661, 277 committed:true 2016-04-07T04:15:20.365-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34385322, 27192661, 277 committed:true) SnapCreateDur 838.060113ms SnapOpenDur 36.048µs 2016-04-07T04:15:20.383-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 837 2016-04-07T04:15:20.407-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27310000 QueuedCount 9174 2016-04-07T04:15:21.663-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.2µs CommitTime 291.474572ms TotalFlushTime 24m47.270782399s TotalCommitTime 2m28.453711502s 2016-04-07T04:15:21.663-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34405356, 27202678, 278 committed:true 2016-04-07T04:15:21.663-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34405356, 27202678, 278 committed:true) SnapCreateDur 303.011641ms SnapOpenDur 41.445µs 2016-04-07T04:15:21.687-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:21.702-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27320000 QueuedCount 8383 2016-04-07T04:15:21.712-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:21.736-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:21.756-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36430000 2016-04-07T04:15:21.76-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:21.783-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:21.798-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36440000 2016-04-07T04:15:21.843-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36450000 2016-04-07T04:15:21.895-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36460000 2016-04-07T04:15:21.941-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36470000 2016-04-07T04:15:22.005-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36480000 2016-04-07T04:15:22.079-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36490000 2016-04-07T04:15:22.138-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36500000 2016-04-07T04:15:22.194-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36510000 2016-04-07T04:15:22.266-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36520000 2016-04-07T04:15:22.341-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36530000 2016-04-07T04:15:22.417-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36540000 2016-04-07T04:15:22.498-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36550000 2016-04-07T04:15:22.689-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27330000 QueuedCount 40251 2016-04-07T04:15:23.559-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27340000 QueuedCount 30251 2016-04-07T04:15:24.507-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27350000 QueuedCount 20251 2016-04-07T04:15:25.384-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27360000 QueuedCount 10251 2016-04-07T04:15:26.216-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27370000 QueuedCount 251 2016-04-07T04:15:26.227-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34505956, 27252978, 0 committed:false 2016-04-07T04:15:26.67-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34505956, 27252978, 0 committed:false) SnapCreateDur 18.188µs SnapOpenDur 442.576449ms 2016-04-07T04:15:27.654-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27380000 QueuedCount 1397 2016-04-07T04:15:28.61-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.551µs CommitTime 803.000559ms TotalFlushTime 24m52.882879182s TotalCommitTime 2m29.256712061s 2016-04-07T04:15:28.61-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34527988, 27263994, 279 committed:true 2016-04-07T04:15:28.61-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34527988, 27263994, 279 committed:true) SnapCreateDur 814.43283ms SnapOpenDur 42.053µs 2016-04-07T04:15:28.635-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:28.658-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:28.683-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:28.707-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:28.731-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:32.155-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27390000 QueuedCount 45331 2016-04-07T04:15:36.022-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 837 2016-04-07T04:15:36.052-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 837 2016-04-07T04:15:36.082-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 837 2016-04-07T04:15:36.112-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 837 2016-04-07T04:15:36.142-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 837 2016-04-07T04:15:36.172-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 837 2016-04-07T04:15:36.202-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 837 2016-04-07T04:15:36.232-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 837 2016-04-07T04:15:36.262-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 837 2016-04-07T04:15:36.292-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 837 2016-04-07T04:15:36.322-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 837 2016-04-07T04:15:36.352-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 837 2016-04-07T04:15:36.382-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 837 2016-04-07T04:15:36.412-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 837 2016-04-07T04:15:36.442-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 837 2016-04-07T04:15:36.472-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 837 2016-04-07T04:15:36.502-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 837 2016-04-07T04:15:36.532-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 837 2016-04-07T04:15:36.562-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 837 2016-04-07T04:15:36.592-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 837 2016-04-07T04:15:36.622-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 837 2016-04-07T04:15:36.652-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 837 2016-04-07T04:15:36.682-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 837 2016-04-07T04:15:36.712-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 837 2016-04-07T04:15:36.742-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 837 2016-04-07T04:15:36.772-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 837 2016-04-07T04:15:36.802-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 837 2016-04-07T04:15:36.832-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 837 2016-04-07T04:15:36.862-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 837 2016-04-07T04:15:36.892-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 837 2016-04-07T04:15:36.898-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27400000 QueuedCount 35331 2016-04-07T04:15:36.922-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 837 2016-04-07T04:15:36.952-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 837 2016-04-07T04:15:36.982-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 837 2016-04-07T04:15:37.012-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 837 2016-04-07T04:15:37.042-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 837 2016-04-07T04:15:37.072-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 837 2016-04-07T04:15:37.102-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 837 2016-04-07T04:15:37.132-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 837 2016-04-07T04:15:37.162-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 837 2016-04-07T04:15:37.192-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 837 2016-04-07T04:15:37.222-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 837 2016-04-07T04:15:37.252-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 837 2016-04-07T04:15:37.282-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 837 2016-04-07T04:15:37.312-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 837 2016-04-07T04:15:37.342-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 837 2016-04-07T04:15:37.372-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 837 2016-04-07T04:15:37.402-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 837 2016-04-07T04:15:37.432-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 837 2016-04-07T04:15:37.462-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 837 2016-04-07T04:15:37.492-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 837 2016-04-07T04:15:37.522-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 837 2016-04-07T04:15:37.552-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 837 2016-04-07T04:15:37.582-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 837 2016-04-07T04:15:37.612-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 837 2016-04-07T04:15:37.642-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 837 2016-04-07T04:15:37.672-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 837 2016-04-07T04:15:37.702-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 837 2016-04-07T04:15:37.732-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 837 2016-04-07T04:15:37.762-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 837 2016-04-07T04:15:37.789-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27410000 QueuedCount 25331 2016-04-07T04:15:37.792-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 837 2016-04-07T04:15:37.822-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 837 2016-04-07T04:15:37.852-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 837 2016-04-07T04:15:37.882-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 837 2016-04-07T04:15:37.912-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 837 2016-04-07T04:15:37.942-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 837 2016-04-07T04:15:37.972-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 837 2016-04-07T04:15:38.002-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 837 2016-04-07T04:15:38.032-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 837 2016-04-07T04:15:38.062-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 837 2016-04-07T04:15:38.092-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 837 2016-04-07T04:15:38.122-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 837 2016-04-07T04:15:38.152-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 837 2016-04-07T04:15:38.182-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 837 2016-04-07T04:15:38.212-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 837 2016-04-07T04:15:38.242-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 837 2016-04-07T04:15:38.272-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 837 2016-04-07T04:15:38.302-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 837 2016-04-07T04:15:38.332-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 837 2016-04-07T04:15:38.362-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 837 2016-04-07T04:15:38.392-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 837 2016-04-07T04:15:38.422-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 837 2016-04-07T04:15:38.452-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 837 2016-04-07T04:15:38.482-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 837 2016-04-07T04:15:38.512-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 837 2016-04-07T04:15:38.542-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 837 2016-04-07T04:15:38.572-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 837 2016-04-07T04:15:38.602-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 837 2016-04-07T04:15:38.632-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 837 2016-04-07T04:15:38.657-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27420000 QueuedCount 15331 2016-04-07T04:15:38.662-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 837 2016-04-07T04:15:38.692-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 837 2016-04-07T04:15:38.722-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 837 2016-04-07T04:15:38.752-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 837 2016-04-07T04:15:38.782-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 837 2016-04-07T04:15:38.812-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 837 2016-04-07T04:15:38.842-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 837 2016-04-07T04:15:38.872-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 837 2016-04-07T04:15:38.902-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 837 2016-04-07T04:15:38.932-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 837 2016-04-07T04:15:38.962-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 837 2016-04-07T04:15:38.992-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 837 2016-04-07T04:15:39.022-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 837 2016-04-07T04:15:39.052-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 837 2016-04-07T04:15:39.082-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 837 2016-04-07T04:15:39.112-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 837 2016-04-07T04:15:39.142-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 837 2016-04-07T04:15:39.172-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 837 2016-04-07T04:15:39.202-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 837 2016-04-07T04:15:39.232-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 837 2016-04-07T04:15:39.262-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 837 2016-04-07T04:15:39.292-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 837 2016-04-07T04:15:39.322-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 837 2016-04-07T04:15:39.352-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 837 2016-04-07T04:15:39.382-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 837 2016-04-07T04:15:39.412-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 837 2016-04-07T04:15:39.442-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 837 2016-04-07T04:15:39.47-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27430000 QueuedCount 5331 2016-04-07T04:15:39.472-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 837 2016-04-07T04:15:39.502-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 837 2016-04-07T04:15:39.532-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 837 2016-04-07T04:15:39.562-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 837 2016-04-07T04:15:39.592-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 837 2016-04-07T04:15:39.622-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 837 2016-04-07T04:15:39.652-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 837 2016-04-07T04:15:39.682-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 837 2016-04-07T04:15:39.712-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8700 Milliseconds Vbucket 837 2016-04-07T04:15:39.742-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8730 Milliseconds Vbucket 837 2016-04-07T04:15:39.772-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8760 Milliseconds Vbucket 837 2016-04-07T04:15:39.802-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8790 Milliseconds Vbucket 837 2016-04-07T04:15:39.832-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8820 Milliseconds Vbucket 837 2016-04-07T04:15:39.862-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8850 Milliseconds Vbucket 837 2016-04-07T04:15:39.892-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8880 Milliseconds Vbucket 837 2016-04-07T04:15:39.922-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8910 Milliseconds Vbucket 837 2016-04-07T04:15:39.952-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8940 Milliseconds Vbucket 837 2016-04-07T04:15:39.964-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34634268, 27317134, 0 committed:false 2016-04-07T04:15:39.982-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8970 Milliseconds Vbucket 837 2016-04-07T04:15:40.012-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9000 Milliseconds Vbucket 837 2016-04-07T04:15:40.042-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9030 Milliseconds Vbucket 837 2016-04-07T04:15:40.072-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9060 Milliseconds Vbucket 837 2016-04-07T04:15:40.102-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9090 Milliseconds Vbucket 837 2016-04-07T04:15:40.132-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9120 Milliseconds Vbucket 837 2016-04-07T04:15:40.162-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9150 Milliseconds Vbucket 837 2016-04-07T04:15:40.192-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9180 Milliseconds Vbucket 837 2016-04-07T04:15:40.222-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9210 Milliseconds Vbucket 837 2016-04-07T04:15:40.252-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9240 Milliseconds Vbucket 837 2016-04-07T04:15:40.282-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9270 Milliseconds Vbucket 837 2016-04-07T04:15:40.312-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9300 Milliseconds Vbucket 837 2016-04-07T04:15:40.342-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9330 Milliseconds Vbucket 837 2016-04-07T04:15:40.372-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9360 Milliseconds Vbucket 837 2016-04-07T04:15:40.402-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9390 Milliseconds Vbucket 837 2016-04-07T04:15:40.425-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34634268, 27317134, 0 committed:false) SnapCreateDur 30.394µs SnapOpenDur 461.315142ms 2016-04-07T04:15:40.432-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9420 Milliseconds Vbucket 837 2016-04-07T04:15:40.826-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27440000 QueuedCount 2616 2016-04-07T04:15:41.896-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.42µs CommitTime 816.436493ms TotalFlushTime 25m4.792391912s TotalCommitTime 2m30.073148554s 2016-04-07T04:15:41.896-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34648596, 27324298, 280 committed:true 2016-04-07T04:15:41.896-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34648596, 27324298, 280 committed:true) SnapCreateDur 826.930719ms SnapOpenDur 41.268µs 2016-04-07T04:15:41.92-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:41.942-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:41.966-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:41.991-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:42.017-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:42.018-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36560000 2016-04-07T04:15:42.06-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36570000 2016-04-07T04:15:42.097-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36580000 2016-04-07T04:15:42.136-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36590000 2016-04-07T04:15:42.181-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36600000 2016-04-07T04:15:42.224-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36610000 2016-04-07T04:15:42.267-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36620000 2016-04-07T04:15:42.316-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36630000 2016-04-07T04:15:42.366-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36640000 2016-04-07T04:15:42.414-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36650000 2016-04-07T04:15:42.465-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36660000 2016-04-07T04:15:42.608-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27450000 QueuedCount 46626 2016-04-07T04:15:43.712-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27460000 QueuedCount 36626 2016-04-07T04:15:45.52-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27470000 QueuedCount 26626 2016-04-07T04:15:47.526-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 837 2016-04-07T04:15:47.556-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 837 2016-04-07T04:15:47.586-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 837 2016-04-07T04:15:47.616-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 837 2016-04-07T04:15:47.646-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 837 2016-04-07T04:15:47.676-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 837 2016-04-07T04:15:47.706-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 837 2016-04-07T04:15:47.736-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 837 2016-04-07T04:15:47.766-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 837 2016-04-07T04:15:47.796-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 837 2016-04-07T04:15:47.826-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 837 2016-04-07T04:15:47.856-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 837 2016-04-07T04:15:47.886-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 837 2016-04-07T04:15:47.916-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 837 2016-04-07T04:15:47.946-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 837 2016-04-07T04:15:47.976-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 837 2016-04-07T04:15:48.006-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 837 2016-04-07T04:15:48.036-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 837 2016-04-07T04:15:48.066-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 837 2016-04-07T04:15:48.096-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 837 2016-04-07T04:15:48.126-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 837 2016-04-07T04:15:48.156-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 837 2016-04-07T04:15:48.186-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 837 2016-04-07T04:15:48.216-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 837 2016-04-07T04:15:48.246-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 837 2016-04-07T04:15:48.276-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 837 2016-04-07T04:15:48.306-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 837 2016-04-07T04:15:48.336-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 837 2016-04-07T04:15:48.366-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 837 2016-04-07T04:15:48.396-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 837 2016-04-07T04:15:48.426-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 837 2016-04-07T04:15:48.456-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 837 2016-04-07T04:15:48.486-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 837 2016-04-07T04:15:48.516-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 837 2016-04-07T04:15:48.528-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27480000 QueuedCount 16626 2016-04-07T04:15:48.546-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 837 2016-04-07T04:15:48.576-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 837 2016-04-07T04:15:48.606-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 837 2016-04-07T04:15:48.636-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 837 2016-04-07T04:15:48.666-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 837 2016-04-07T04:15:48.696-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 837 2016-04-07T04:15:48.726-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 837 2016-04-07T04:15:48.756-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 837 2016-04-07T04:15:48.786-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 837 2016-04-07T04:15:48.816-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 837 2016-04-07T04:15:48.846-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 837 2016-04-07T04:15:48.876-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 837 2016-04-07T04:15:48.906-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 837 2016-04-07T04:15:48.936-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 837 2016-04-07T04:15:48.966-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 837 2016-04-07T04:15:48.996-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 837 2016-04-07T04:15:49.026-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 837 2016-04-07T04:15:49.056-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 837 2016-04-07T04:15:49.086-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 837 2016-04-07T04:15:49.116-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 837 2016-04-07T04:15:49.146-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 837 2016-04-07T04:15:49.176-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 837 2016-04-07T04:15:49.206-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 837 2016-04-07T04:15:49.236-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 837 2016-04-07T04:15:49.266-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 837 2016-04-07T04:15:49.296-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 837 2016-04-07T04:15:49.326-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 837 2016-04-07T04:15:49.356-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 837 2016-04-07T04:15:49.386-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 837 2016-04-07T04:15:49.416-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 837 2016-04-07T04:15:49.434-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27490000 QueuedCount 6626 2016-04-07T04:15:49.446-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 837 2016-04-07T04:15:49.476-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 837 2016-04-07T04:15:49.506-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 837 2016-04-07T04:15:49.536-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 837 2016-04-07T04:15:49.566-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 837 2016-04-07T04:15:49.596-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 837 2016-04-07T04:15:49.626-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 837 2016-04-07T04:15:49.656-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 837 2016-04-07T04:15:49.686-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 837 2016-04-07T04:15:49.716-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 837 2016-04-07T04:15:49.746-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 837 2016-04-07T04:15:49.776-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 837 2016-04-07T04:15:49.806-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 837 2016-04-07T04:15:49.836-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 837 2016-04-07T04:15:49.866-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 837 2016-04-07T04:15:49.896-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 837 2016-04-07T04:15:49.926-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 837 2016-04-07T04:15:49.956-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 837 2016-04-07T04:15:49.986-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 837 2016-04-07T04:15:49.986-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34754976, 27377488, 0 committed:false 2016-04-07T04:15:50.016-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 837 2016-04-07T04:15:50.046-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 837 2016-04-07T04:15:50.076-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 837 2016-04-07T04:15:50.106-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 837 2016-04-07T04:15:50.136-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 837 2016-04-07T04:15:50.166-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 837 2016-04-07T04:15:50.196-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 837 2016-04-07T04:15:50.226-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 837 2016-04-07T04:15:50.256-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 837 2016-04-07T04:15:50.286-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 837 2016-04-07T04:15:50.316-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 837 2016-04-07T04:15:50.346-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 837 2016-04-07T04:15:50.376-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 837 2016-04-07T04:15:50.406-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 837 2016-04-07T04:15:50.434-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34754976, 27377488, 0 committed:false) SnapCreateDur 14.001µs SnapOpenDur 448.487051ms 2016-04-07T04:15:50.436-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 837 2016-04-07T04:15:50.837-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27500000 QueuedCount 5525 2016-04-07T04:15:52.081-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.594µs CommitTime 751.265253ms TotalFlushTime 25m13.683044049s TotalCommitTime 2m30.824413807s 2016-04-07T04:15:52.081-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34772464, 27386232, 281 committed:true 2016-04-07T04:15:52.081-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34772464, 27386232, 281 committed:true) SnapCreateDur 762.434366ms SnapOpenDur 64.656µs 2016-04-07T04:15:52.107-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:52.132-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:52.157-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:52.18-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36670000 2016-04-07T04:15:52.182-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:52.206-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:15:52.539-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27510000 QueuedCount 50000 2016-04-07T04:15:53.445-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27520000 QueuedCount 40925 2016-04-07T04:15:54.382-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27530000 QueuedCount 30925 2016-04-07T04:15:56.207-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27540000 QueuedCount 20925 2016-04-07T04:15:57.192-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27550000 QueuedCount 10925 2016-04-07T04:15:57.258-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 837 2016-04-07T04:15:57.288-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 837 2016-04-07T04:15:57.318-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 837 2016-04-07T04:15:57.348-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 837 2016-04-07T04:15:57.378-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 837 2016-04-07T04:15:57.408-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 837 2016-04-07T04:15:57.438-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 837 2016-04-07T04:15:57.468-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 837 2016-04-07T04:15:57.498-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 837 2016-04-07T04:15:57.528-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 837 2016-04-07T04:15:57.558-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 837 2016-04-07T04:15:57.588-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 837 2016-04-07T04:15:57.618-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 837 2016-04-07T04:15:57.648-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 837 2016-04-07T04:15:57.678-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 837 2016-04-07T04:15:57.708-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 837 2016-04-07T04:15:57.738-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 837 2016-04-07T04:15:57.768-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 837 2016-04-07T04:15:57.798-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 837 2016-04-07T04:15:57.828-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 837 2016-04-07T04:15:57.858-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 837 2016-04-07T04:15:57.888-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 837 2016-04-07T04:15:57.918-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 837 2016-04-07T04:15:57.948-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 837 2016-04-07T04:15:57.978-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 837 2016-04-07T04:15:58.008-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 837 2016-04-07T04:15:58.038-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 837 2016-04-07T04:15:58.068-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 837 2016-04-07T04:15:58.098-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 837 2016-04-07T04:15:58.128-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 837 2016-04-07T04:15:58.158-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 837 2016-04-07T04:15:58.182-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27560000 QueuedCount 925 2016-04-07T04:15:58.188-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 837 2016-04-07T04:15:58.208-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34881516, 27440758, 0 committed:false 2016-04-07T04:15:58.218-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 837 2016-04-07T04:15:58.248-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 837 2016-04-07T04:15:58.278-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 837 2016-04-07T04:15:58.308-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 837 2016-04-07T04:15:58.338-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 837 2016-04-07T04:15:58.368-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 837 2016-04-07T04:15:58.398-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 837 2016-04-07T04:15:58.428-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 837 2016-04-07T04:15:58.458-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 837 2016-04-07T04:15:58.488-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 837 2016-04-07T04:15:58.518-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 837 2016-04-07T04:15:58.548-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 837 2016-04-07T04:15:58.578-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 837 2016-04-07T04:15:58.608-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 837 2016-04-07T04:15:58.638-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 837 2016-04-07T04:15:58.668-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 837 2016-04-07T04:15:58.682-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34881516, 27440758, 0 committed:false) SnapCreateDur 21.906µs SnapOpenDur 473.8399ms 2016-04-07T04:15:58.698-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 837 2016-04-07T04:15:58.767-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36680000 2016-04-07T04:15:59.523-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27570000 QueuedCount 229 2016-04-07T04:16:00.303-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.233µs CommitTime 765.150547ms TotalFlushTime 25m20.563962537s TotalCommitTime 2m31.589564354s 2016-04-07T04:16:00.303-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34899822, 27449911, 282 committed:true 2016-04-07T04:16:00.303-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34899822, 27449911, 282 committed:true) SnapCreateDur 776.17066ms SnapOpenDur 34.274µs 2016-04-07T04:16:00.328-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:00.352-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:00.376-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:00.4-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:00.405-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36690000 2016-04-07T04:16:00.424-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:00.447-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36700000 2016-04-07T04:16:00.493-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36710000 2016-04-07T04:16:01.353-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27580000 QueuedCount 35371 2016-04-07T04:16:02.26-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27590000 QueuedCount 25371 2016-04-07T04:16:03.111-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27600000 QueuedCount 15371 2016-04-07T04:16:04.159-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27610000 QueuedCount 5371 2016-04-07T04:16:04.67-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 34988654, 27494327, 0 committed:false 2016-04-07T04:16:05.064-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 34988654, 27494327, 0 committed:false) SnapCreateDur 27.53µs SnapOpenDur 394.778185ms 2016-04-07T04:16:05.099-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36720000 2016-04-07T04:16:05.474-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27620000 QueuedCount 5487 2016-04-07T04:16:06.707-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.592µs CommitTime 736.041533ms TotalFlushTime 25m25.743803379s TotalCommitTime 2m32.325605887s 2016-04-07T04:16:06.707-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35008564, 27504282, 283 committed:true 2016-04-07T04:16:06.707-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35008564, 27504282, 283 committed:true) SnapCreateDur 747.064989ms SnapOpenDur 33.134µs 2016-04-07T04:16:06.732-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:06.757-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:06.782-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:06.807-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:06.83-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:07.14-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27630000 QueuedCount 45960 2016-04-07T04:16:08.123-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27640000 QueuedCount 35960 2016-04-07T04:16:11.889-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 837 2016-04-07T04:16:11.919-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 837 2016-04-07T04:16:11.949-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 837 2016-04-07T04:16:11.979-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 837 2016-04-07T04:16:12.009-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 837 2016-04-07T04:16:12.039-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 837 2016-04-07T04:16:12.069-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 837 2016-04-07T04:16:12.099-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 837 2016-04-07T04:16:12.129-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 837 2016-04-07T04:16:12.159-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 837 2016-04-07T04:16:12.189-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 837 2016-04-07T04:16:12.219-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 837 2016-04-07T04:16:12.249-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 837 2016-04-07T04:16:12.279-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 837 2016-04-07T04:16:12.309-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 837 2016-04-07T04:16:12.339-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 837 2016-04-07T04:16:12.369-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 837 2016-04-07T04:16:12.399-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 837 2016-04-07T04:16:12.429-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 837 2016-04-07T04:16:12.459-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 837 2016-04-07T04:16:12.489-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 837 2016-04-07T04:16:12.519-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 837 2016-04-07T04:16:12.549-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 837 2016-04-07T04:16:12.579-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 837 2016-04-07T04:16:12.609-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 837 2016-04-07T04:16:12.639-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 837 2016-04-07T04:16:12.669-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 837 2016-04-07T04:16:12.699-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 837 2016-04-07T04:16:12.729-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 837 2016-04-07T04:16:12.759-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 837 2016-04-07T04:16:12.789-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 837 2016-04-07T04:16:12.819-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 837 2016-04-07T04:16:12.849-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 837 2016-04-07T04:16:12.879-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 837 2016-04-07T04:16:12.909-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 837 2016-04-07T04:16:12.939-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 837 2016-04-07T04:16:12.951-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":9052779,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":2882405107,"bucket-1:myindex:avg_ts_items_count":22456,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":7960414680,"bucket-1:myindex:delete_bytes":323626643,"bucket-1:myindex:disk_size":72118333440,"bucket-1:myindex:flush_queue_size":26151,"bucket-1:myindex:frag_percent":88,"bucket-1:myindex:get_bytes":328853422,"bucket-1:myindex:insert_bytes":2890251062,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":283,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":28125348,"bucket-1:myindex:num_docs_pending":59300,"bucket-1:myindex:num_docs_queued":11815352,"bucket-1:myindex:num_flush_queued":27675960,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":413,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1642491956,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"283 152325605887 2637486922394514207","bucket-1:myindex:timings/storage_del":"7528222 213052323826 -7312506156762182248","bucket-1:myindex:timings/storage_get":"27649811 204508560003 751262765427731575","bucket-1:myindex:timings/storage_handle_open":"283 6807319 174580622897","bucket-1:myindex:timings/storage_info":"1393 12308644 302583527378","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"286 192524691 3683943303028401","bucket-1:myindex:timings/storage_meta_set":"283 446660650 107866416717519480","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"55056442 1096387267018 4203149156109766986","bucket-1:myindex:timings/storage_snapshot_close":"130 5226739512 221748901432575102","bucket-1:myindex:timings/storage_snapshot_create":"130 48722076705 9093769050237334339","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36728739,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":764,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"5293 808010083 144420836530293"} 2016-04-07T04:16:12.955-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27650000 QueuedCount 25960 2016-04-07T04:16:12.969-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 837 2016-04-07T04:16:12.999-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 837 2016-04-07T04:16:13.029-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 837 2016-04-07T04:16:13.059-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 837 2016-04-07T04:16:13.089-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 837 2016-04-07T04:16:13.119-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 837 2016-04-07T04:16:13.149-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 837 2016-04-07T04:16:13.179-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 837 2016-04-07T04:16:13.209-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 837 2016-04-07T04:16:13.239-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 837 2016-04-07T04:16:13.269-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 837 2016-04-07T04:16:13.299-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 837 2016-04-07T04:16:13.329-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 837 2016-04-07T04:16:13.359-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 837 2016-04-07T04:16:13.389-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 837 2016-04-07T04:16:13.419-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 837 2016-04-07T04:16:13.449-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 837 2016-04-07T04:16:13.479-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 837 2016-04-07T04:16:13.509-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 837 2016-04-07T04:16:13.539-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 837 2016-04-07T04:16:13.569-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 837 2016-04-07T04:16:13.599-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 837 2016-04-07T04:16:13.629-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 837 2016-04-07T04:16:13.659-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 837 2016-04-07T04:16:13.689-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 837 2016-04-07T04:16:13.719-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 837 2016-04-07T04:16:13.749-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 837 2016-04-07T04:16:13.779-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 837 2016-04-07T04:16:13.809-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 837 2016-04-07T04:16:13.839-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 837 2016-04-07T04:16:13.869-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 837 2016-04-07T04:16:13.899-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 837 2016-04-07T04:16:13.912-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27660000 QueuedCount 15960 2016-04-07T04:16:13.929-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 837 2016-04-07T04:16:13.959-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 837 2016-04-07T04:16:13.989-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 837 2016-04-07T04:16:14.019-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 837 2016-04-07T04:16:14.049-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 837 2016-04-07T04:16:14.079-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 837 2016-04-07T04:16:14.109-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 837 2016-04-07T04:16:14.139-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 837 2016-04-07T04:16:14.169-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 837 2016-04-07T04:16:14.199-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 837 2016-04-07T04:16:14.229-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 837 2016-04-07T04:16:14.259-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 837 2016-04-07T04:16:14.289-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 837 2016-04-07T04:16:14.319-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 837 2016-04-07T04:16:14.349-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 837 2016-04-07T04:16:14.379-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 837 2016-04-07T04:16:14.409-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 837 2016-04-07T04:16:14.439-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 837 2016-04-07T04:16:14.469-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 837 2016-04-07T04:16:14.499-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 837 2016-04-07T04:16:14.529-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 837 2016-04-07T04:16:14.559-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 837 2016-04-07T04:16:14.589-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 837 2016-04-07T04:16:14.619-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 837 2016-04-07T04:16:14.649-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 837 2016-04-07T04:16:14.679-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 837 2016-04-07T04:16:14.709-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 837 2016-04-07T04:16:14.739-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 837 2016-04-07T04:16:14.768-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27670000 QueuedCount 5960 2016-04-07T04:16:14.769-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 837 2016-04-07T04:16:14.799-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 837 2016-04-07T04:16:14.829-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 837 2016-04-07T04:16:14.859-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 837 2016-04-07T04:16:14.889-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 837 2016-04-07T04:16:14.919-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 837 2016-04-07T04:16:14.949-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 837 2016-04-07T04:16:14.979-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 837 2016-04-07T04:16:15.009-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 837 2016-04-07T04:16:15.039-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 837 2016-04-07T04:16:15.069-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 837 2016-04-07T04:16:15.099-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 837 2016-04-07T04:16:15.129-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 837 2016-04-07T04:16:15.159-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 837 2016-04-07T04:16:15.189-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 837 2016-04-07T04:16:15.219-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 837 2016-04-07T04:16:15.249-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 837 2016-04-07T04:16:15.279-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 837 2016-04-07T04:16:15.309-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 837 2016-04-07T04:16:15.339-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 837 2016-04-07T04:16:15.369-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 837 2016-04-07T04:16:15.373-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35107942, 27553971, 0 committed:false 2016-04-07T04:16:15.399-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 837 2016-04-07T04:16:15.429-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 837 2016-04-07T04:16:15.459-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 837 2016-04-07T04:16:15.489-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 837 2016-04-07T04:16:15.519-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 837 2016-04-07T04:16:15.549-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 837 2016-04-07T04:16:15.579-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8700 Milliseconds Vbucket 837 2016-04-07T04:16:15.609-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8730 Milliseconds Vbucket 837 2016-04-07T04:16:15.639-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8760 Milliseconds Vbucket 837 2016-04-07T04:16:15.669-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8790 Milliseconds Vbucket 837 2016-04-07T04:16:15.699-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8820 Milliseconds Vbucket 837 2016-04-07T04:16:15.729-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8850 Milliseconds Vbucket 837 2016-04-07T04:16:15.759-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8880 Milliseconds Vbucket 837 2016-04-07T04:16:15.789-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8910 Milliseconds Vbucket 837 2016-04-07T04:16:15.806-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35107942, 27553971, 0 committed:false) SnapCreateDur 9.603µs SnapOpenDur 433.939826ms 2016-04-07T04:16:15.819-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8940 Milliseconds Vbucket 837 2016-04-07T04:16:16.214-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27680000 QueuedCount 3546 2016-04-07T04:16:17.256-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.573µs CommitTime 764.71931ms TotalFlushTime 25m35.009594798s TotalCommitTime 2m33.090325197s 2016-04-07T04:16:17.257-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35122860, 27561430, 284 committed:true 2016-04-07T04:16:17.257-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35122860, 27561430, 284 committed:true) SnapCreateDur 775.60988ms SnapOpenDur 36.082µs 2016-04-07T04:16:17.282-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:17.309-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:17.327-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36730000 2016-04-07T04:16:17.334-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:17.359-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:17.565-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:17.834-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27690000 QueuedCount 50000 2016-04-07T04:16:18.773-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27700000 QueuedCount 46620 2016-04-07T04:16:19.788-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27710000 QueuedCount 36620 2016-04-07T04:16:23.245-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5010 Milliseconds Vbucket 837 2016-04-07T04:16:23.275-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5040 Milliseconds Vbucket 837 2016-04-07T04:16:23.305-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5070 Milliseconds Vbucket 837 2016-04-07T04:16:23.335-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5100 Milliseconds Vbucket 837 2016-04-07T04:16:23.365-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5130 Milliseconds Vbucket 837 2016-04-07T04:16:23.395-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5160 Milliseconds Vbucket 837 2016-04-07T04:16:23.425-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5190 Milliseconds Vbucket 837 2016-04-07T04:16:23.455-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5220 Milliseconds Vbucket 837 2016-04-07T04:16:23.485-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5250 Milliseconds Vbucket 837 2016-04-07T04:16:23.515-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5280 Milliseconds Vbucket 837 2016-04-07T04:16:23.545-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5310 Milliseconds Vbucket 837 2016-04-07T04:16:23.575-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5340 Milliseconds Vbucket 837 2016-04-07T04:16:23.605-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5370 Milliseconds Vbucket 837 2016-04-07T04:16:23.635-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5400 Milliseconds Vbucket 837 2016-04-07T04:16:23.665-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5430 Milliseconds Vbucket 837 2016-04-07T04:16:23.695-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5460 Milliseconds Vbucket 837 2016-04-07T04:16:23.725-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5490 Milliseconds Vbucket 837 2016-04-07T04:16:23.755-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5520 Milliseconds Vbucket 837 2016-04-07T04:16:23.785-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5550 Milliseconds Vbucket 837 2016-04-07T04:16:23.815-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5580 Milliseconds Vbucket 837 2016-04-07T04:16:23.845-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5610 Milliseconds Vbucket 837 2016-04-07T04:16:23.875-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5640 Milliseconds Vbucket 837 2016-04-07T04:16:23.905-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5670 Milliseconds Vbucket 837 2016-04-07T04:16:23.935-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5700 Milliseconds Vbucket 837 2016-04-07T04:16:23.965-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5730 Milliseconds Vbucket 837 2016-04-07T04:16:23.995-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5760 Milliseconds Vbucket 837 2016-04-07T04:16:24.025-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5790 Milliseconds Vbucket 837 2016-04-07T04:16:24.055-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5820 Milliseconds Vbucket 837 2016-04-07T04:16:24.085-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5850 Milliseconds Vbucket 837 2016-04-07T04:16:24.115-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5880 Milliseconds Vbucket 837 2016-04-07T04:16:24.145-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5910 Milliseconds Vbucket 837 2016-04-07T04:16:24.175-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5940 Milliseconds Vbucket 837 2016-04-07T04:16:24.205-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 5970 Milliseconds Vbucket 837 2016-04-07T04:16:24.235-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6000 Milliseconds Vbucket 837 2016-04-07T04:16:24.265-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6030 Milliseconds Vbucket 837 2016-04-07T04:16:24.295-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6060 Milliseconds Vbucket 837 2016-04-07T04:16:24.325-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6090 Milliseconds Vbucket 837 2016-04-07T04:16:24.355-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6120 Milliseconds Vbucket 837 2016-04-07T04:16:24.385-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6150 Milliseconds Vbucket 837 2016-04-07T04:16:24.415-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6180 Milliseconds Vbucket 837 2016-04-07T04:16:24.445-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6210 Milliseconds Vbucket 837 2016-04-07T04:16:24.475-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6240 Milliseconds Vbucket 837 2016-04-07T04:16:24.505-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6270 Milliseconds Vbucket 837 2016-04-07T04:16:24.535-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6300 Milliseconds Vbucket 837 2016-04-07T04:16:24.565-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6330 Milliseconds Vbucket 837 2016-04-07T04:16:24.595-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6360 Milliseconds Vbucket 837 2016-04-07T04:16:24.625-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6390 Milliseconds Vbucket 837 2016-04-07T04:16:24.655-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6420 Milliseconds Vbucket 837 2016-04-07T04:16:24.685-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6450 Milliseconds Vbucket 837 2016-04-07T04:16:24.715-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6480 Milliseconds Vbucket 837 2016-04-07T04:16:24.745-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6510 Milliseconds Vbucket 837 2016-04-07T04:16:24.775-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6540 Milliseconds Vbucket 837 2016-04-07T04:16:24.805-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6570 Milliseconds Vbucket 837 2016-04-07T04:16:24.835-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6600 Milliseconds Vbucket 837 2016-04-07T04:16:24.865-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6630 Milliseconds Vbucket 837 2016-04-07T04:16:24.895-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6660 Milliseconds Vbucket 837 2016-04-07T04:16:24.925-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6690 Milliseconds Vbucket 837 2016-04-07T04:16:24.955-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6720 Milliseconds Vbucket 837 2016-04-07T04:16:24.985-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6750 Milliseconds Vbucket 837 2016-04-07T04:16:25.015-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6780 Milliseconds Vbucket 837 2016-04-07T04:16:25.045-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6810 Milliseconds Vbucket 837 2016-04-07T04:16:25.075-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6840 Milliseconds Vbucket 837 2016-04-07T04:16:25.105-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6870 Milliseconds Vbucket 837 2016-04-07T04:16:25.135-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6900 Milliseconds Vbucket 837 2016-04-07T04:16:25.165-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6930 Milliseconds Vbucket 837 2016-04-07T04:16:25.195-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6960 Milliseconds Vbucket 837 2016-04-07T04:16:25.225-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 6990 Milliseconds Vbucket 837 2016-04-07T04:16:25.255-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7020 Milliseconds Vbucket 837 2016-04-07T04:16:25.285-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7050 Milliseconds Vbucket 837 2016-04-07T04:16:25.315-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7080 Milliseconds Vbucket 837 2016-04-07T04:16:25.345-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7110 Milliseconds Vbucket 837 2016-04-07T04:16:25.375-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7140 Milliseconds Vbucket 837 2016-04-07T04:16:25.405-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7170 Milliseconds Vbucket 837 2016-04-07T04:16:25.435-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7200 Milliseconds Vbucket 837 2016-04-07T04:16:25.465-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7230 Milliseconds Vbucket 837 2016-04-07T04:16:25.495-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7260 Milliseconds Vbucket 837 2016-04-07T04:16:25.525-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7290 Milliseconds Vbucket 837 2016-04-07T04:16:25.555-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7320 Milliseconds Vbucket 837 2016-04-07T04:16:25.585-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7350 Milliseconds Vbucket 837 2016-04-07T04:16:25.615-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7380 Milliseconds Vbucket 837 2016-04-07T04:16:25.645-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7410 Milliseconds Vbucket 837 2016-04-07T04:16:25.675-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7440 Milliseconds Vbucket 837 2016-04-07T04:16:25.705-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7470 Milliseconds Vbucket 837 2016-04-07T04:16:25.735-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7500 Milliseconds Vbucket 837 2016-04-07T04:16:25.765-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7530 Milliseconds Vbucket 837 2016-04-07T04:16:25.795-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7560 Milliseconds Vbucket 837 2016-04-07T04:16:25.825-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7590 Milliseconds Vbucket 837 2016-04-07T04:16:25.855-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7620 Milliseconds Vbucket 837 2016-04-07T04:16:25.885-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7650 Milliseconds Vbucket 837 2016-04-07T04:16:25.915-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7680 Milliseconds Vbucket 837 2016-04-07T04:16:25.945-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7710 Milliseconds Vbucket 837 2016-04-07T04:16:25.975-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7740 Milliseconds Vbucket 837 2016-04-07T04:16:26.005-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7770 Milliseconds Vbucket 837 2016-04-07T04:16:26.035-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7800 Milliseconds Vbucket 837 2016-04-07T04:16:26.065-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7830 Milliseconds Vbucket 837 2016-04-07T04:16:26.095-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7860 Milliseconds Vbucket 837 2016-04-07T04:16:26.125-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7890 Milliseconds Vbucket 837 2016-04-07T04:16:26.155-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7920 Milliseconds Vbucket 837 2016-04-07T04:16:26.185-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7950 Milliseconds Vbucket 837 2016-04-07T04:16:26.215-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 7980 Milliseconds Vbucket 837 2016-04-07T04:16:26.245-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8010 Milliseconds Vbucket 837 2016-04-07T04:16:26.275-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8040 Milliseconds Vbucket 837 2016-04-07T04:16:26.305-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8070 Milliseconds Vbucket 837 2016-04-07T04:16:26.335-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8100 Milliseconds Vbucket 837 2016-04-07T04:16:26.365-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8130 Milliseconds Vbucket 837 2016-04-07T04:16:26.395-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8160 Milliseconds Vbucket 837 2016-04-07T04:16:26.425-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8190 Milliseconds Vbucket 837 2016-04-07T04:16:26.455-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8220 Milliseconds Vbucket 837 2016-04-07T04:16:26.485-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8250 Milliseconds Vbucket 837 2016-04-07T04:16:26.515-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8280 Milliseconds Vbucket 837 2016-04-07T04:16:26.545-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8310 Milliseconds Vbucket 837 2016-04-07T04:16:26.575-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8340 Milliseconds Vbucket 837 2016-04-07T04:16:26.605-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8370 Milliseconds Vbucket 837 2016-04-07T04:16:26.635-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8400 Milliseconds Vbucket 837 2016-04-07T04:16:26.659-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27720000 QueuedCount 26620 2016-04-07T04:16:26.665-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8430 Milliseconds Vbucket 837 2016-04-07T04:16:26.695-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8460 Milliseconds Vbucket 837 2016-04-07T04:16:26.725-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8490 Milliseconds Vbucket 837 2016-04-07T04:16:26.755-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8520 Milliseconds Vbucket 837 2016-04-07T04:16:26.785-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8550 Milliseconds Vbucket 837 2016-04-07T04:16:26.815-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8580 Milliseconds Vbucket 837 2016-04-07T04:16:26.845-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8610 Milliseconds Vbucket 837 2016-04-07T04:16:26.875-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8640 Milliseconds Vbucket 837 2016-04-07T04:16:26.905-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8670 Milliseconds Vbucket 837 2016-04-07T04:16:26.935-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8700 Milliseconds Vbucket 837 2016-04-07T04:16:26.965-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8730 Milliseconds Vbucket 837 2016-04-07T04:16:26.995-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8760 Milliseconds Vbucket 837 2016-04-07T04:16:27.025-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8790 Milliseconds Vbucket 837 2016-04-07T04:16:27.055-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8820 Milliseconds Vbucket 837 2016-04-07T04:16:27.085-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8850 Milliseconds Vbucket 837 2016-04-07T04:16:27.115-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8880 Milliseconds Vbucket 837 2016-04-07T04:16:27.145-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8910 Milliseconds Vbucket 837 2016-04-07T04:16:27.175-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8940 Milliseconds Vbucket 837 2016-04-07T04:16:27.205-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 8970 Milliseconds Vbucket 837 2016-04-07T04:16:27.235-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9000 Milliseconds Vbucket 837 2016-04-07T04:16:27.265-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9030 Milliseconds Vbucket 837 2016-04-07T04:16:27.295-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9060 Milliseconds Vbucket 837 2016-04-07T04:16:27.325-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9090 Milliseconds Vbucket 837 2016-04-07T04:16:27.355-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9120 Milliseconds Vbucket 837 2016-04-07T04:16:27.385-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9150 Milliseconds Vbucket 837 2016-04-07T04:16:27.415-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9180 Milliseconds Vbucket 837 2016-04-07T04:16:27.445-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9210 Milliseconds Vbucket 837 2016-04-07T04:16:27.475-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9240 Milliseconds Vbucket 837 2016-04-07T04:16:27.476-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27730000 QueuedCount 16620 2016-04-07T04:16:27.505-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9270 Milliseconds Vbucket 837 2016-04-07T04:16:27.535-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9300 Milliseconds Vbucket 837 2016-04-07T04:16:27.565-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9330 Milliseconds Vbucket 837 2016-04-07T04:16:27.595-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9360 Milliseconds Vbucket 837 2016-04-07T04:16:27.625-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9390 Milliseconds Vbucket 837 2016-04-07T04:16:27.655-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9420 Milliseconds Vbucket 837 2016-04-07T04:16:27.685-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9450 Milliseconds Vbucket 837 2016-04-07T04:16:27.715-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9480 Milliseconds Vbucket 837 2016-04-07T04:16:27.745-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9510 Milliseconds Vbucket 837 2016-04-07T04:16:27.775-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9540 Milliseconds Vbucket 837 2016-04-07T04:16:27.805-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9570 Milliseconds Vbucket 837 2016-04-07T04:16:27.835-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9600 Milliseconds Vbucket 837 2016-04-07T04:16:27.865-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9630 Milliseconds Vbucket 837 2016-04-07T04:16:27.895-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9660 Milliseconds Vbucket 837 2016-04-07T04:16:27.925-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9690 Milliseconds Vbucket 837 2016-04-07T04:16:27.955-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9720 Milliseconds Vbucket 837 2016-04-07T04:16:27.985-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9750 Milliseconds Vbucket 837 2016-04-07T04:16:28.015-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9780 Milliseconds Vbucket 837 2016-04-07T04:16:28.045-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9810 Milliseconds Vbucket 837 2016-04-07T04:16:28.075-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9840 Milliseconds Vbucket 837 2016-04-07T04:16:28.105-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9870 Milliseconds Vbucket 837 2016-04-07T04:16:28.135-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9900 Milliseconds Vbucket 837 2016-04-07T04:16:28.165-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9930 Milliseconds Vbucket 837 2016-04-07T04:16:28.195-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9960 Milliseconds Vbucket 837 2016-04-07T04:16:28.225-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 9990 Milliseconds Vbucket 837 2016-04-07T04:16:28.255-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10020 Milliseconds Vbucket 837 2016-04-07T04:16:28.285-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10050 Milliseconds Vbucket 837 2016-04-07T04:16:28.315-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10080 Milliseconds Vbucket 837 2016-04-07T04:16:28.345-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10110 Milliseconds Vbucket 837 2016-04-07T04:16:28.351-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27740000 QueuedCount 6620 2016-04-07T04:16:28.375-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10140 Milliseconds Vbucket 837 2016-04-07T04:16:28.405-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10170 Milliseconds Vbucket 837 2016-04-07T04:16:28.435-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10200 Milliseconds Vbucket 837 2016-04-07T04:16:28.465-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10230 Milliseconds Vbucket 837 2016-04-07T04:16:28.495-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10260 Milliseconds Vbucket 837 2016-04-07T04:16:28.525-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10290 Milliseconds Vbucket 837 2016-04-07T04:16:28.555-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10320 Milliseconds Vbucket 837 2016-04-07T04:16:28.585-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10350 Milliseconds Vbucket 837 2016-04-07T04:16:28.615-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10380 Milliseconds Vbucket 837 2016-04-07T04:16:28.645-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10410 Milliseconds Vbucket 837 2016-04-07T04:16:28.675-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10440 Milliseconds Vbucket 837 2016-04-07T04:16:28.705-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10470 Milliseconds Vbucket 837 2016-04-07T04:16:28.735-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10500 Milliseconds Vbucket 837 2016-04-07T04:16:28.765-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10530 Milliseconds Vbucket 837 2016-04-07T04:16:28.795-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10560 Milliseconds Vbucket 837 2016-04-07T04:16:28.825-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10590 Milliseconds Vbucket 837 2016-04-07T04:16:28.855-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10620 Milliseconds Vbucket 837 2016-04-07T04:16:28.885-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10650 Milliseconds Vbucket 837 2016-04-07T04:16:28.915-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10680 Milliseconds Vbucket 837 2016-04-07T04:16:28.945-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10710 Milliseconds Vbucket 837 2016-04-07T04:16:28.964-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35246904, 27623452, 0 committed:false 2016-04-07T04:16:28.975-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10740 Milliseconds Vbucket 837 2016-04-07T04:16:29.005-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10770 Milliseconds Vbucket 837 2016-04-07T04:16:29.035-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10800 Milliseconds Vbucket 837 2016-04-07T04:16:29.065-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10830 Milliseconds Vbucket 837 2016-04-07T04:16:29.095-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10860 Milliseconds Vbucket 837 2016-04-07T04:16:29.125-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10890 Milliseconds Vbucket 837 2016-04-07T04:16:29.155-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10920 Milliseconds Vbucket 837 2016-04-07T04:16:29.185-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10950 Milliseconds Vbucket 837 2016-04-07T04:16:29.215-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 10980 Milliseconds Vbucket 837 2016-04-07T04:16:29.245-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11010 Milliseconds Vbucket 837 2016-04-07T04:16:29.275-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11040 Milliseconds Vbucket 837 2016-04-07T04:16:29.305-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11070 Milliseconds Vbucket 837 2016-04-07T04:16:29.335-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11100 Milliseconds Vbucket 837 2016-04-07T04:16:29.365-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11130 Milliseconds Vbucket 837 2016-04-07T04:16:29.395-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11160 Milliseconds Vbucket 837 2016-04-07T04:16:29.425-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11190 Milliseconds Vbucket 837 2016-04-07T04:16:29.438-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35246904, 27623452, 0 committed:false) SnapCreateDur 10.694µs SnapOpenDur 474.051284ms 2016-04-07T04:16:29.455-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 11220 Milliseconds Vbucket 837 2016-04-07T04:16:29.713-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27750000 QueuedCount 6161 2016-04-07T04:16:31.068-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.903µs CommitTime 863.154409ms TotalFlushTime 25m47.370738455s TotalCommitTime 2m33.953479606s 2016-04-07T04:16:31.068-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35265684, 27632842, 285 committed:true 2016-04-07T04:16:31.068-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35265684, 27632842, 285 committed:true) SnapCreateDur 874.12646ms SnapOpenDur 35.916µs 2016-04-07T04:16:31.093-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:31.119-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:31.143-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:31.168-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:31.192-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:31.382-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27760000 QueuedCount 47719 2016-04-07T04:16:32.299-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27770000 QueuedCount 37719 2016-04-07T04:16:33.268-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27780000 QueuedCount 27719 2016-04-07T04:16:34.188-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27790000 QueuedCount 17719 2016-04-07T04:16:35.074-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27800000 QueuedCount 7719 2016-04-07T04:16:35.671-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35367224, 27683612, 0 committed:false 2016-04-07T04:16:36.032-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35367224, 27683612, 0 committed:false) SnapCreateDur 12.384µs SnapOpenDur 361.056227ms 2016-04-07T04:16:36.062-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36740000 2016-04-07T04:16:36.103-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36750000 2016-04-07T04:16:36.143-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36760000 2016-04-07T04:16:36.185-07:00 [Info] logReaderStat:: MAINT_STREAM MutationCount 36770000 2016-04-07T04:16:36.295-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27810000 QueuedCount 4155 2016-04-07T04:16:37.435-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.908µs CommitTime 768.979846ms TotalFlushTime 25m52.514120089s TotalCommitTime 2m34.722459452s 2016-04-07T04:16:37.435-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35379882, 27689941, 286 committed:true 2016-04-07T04:16:37.435-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35379882, 27689941, 286 committed:true) SnapCreateDur 779.36399ms SnapOpenDur 42.159µs 2016-04-07T04:16:37.459-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:37.484-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:37.509-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:37.535-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:37.56-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:39.9-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27820000 QueuedCount 49273 2016-04-07T04:16:40.875-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27830000 QueuedCount 39273 2016-04-07T04:16:41.981-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27840000 QueuedCount 29273 2016-04-07T04:16:43.741-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27850000 QueuedCount 19273 2016-04-07T04:16:44.53-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27860000 QueuedCount 9273 2016-04-07T04:16:45.349-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35488450, 27744225, 0 committed:false 2016-04-07T04:16:45.718-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35488450, 27744225, 0 committed:false) SnapCreateDur 34.897µs SnapOpenDur 368.979097ms 2016-04-07T04:16:45.754-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27870000 QueuedCount 9763 2016-04-07T04:16:51.694-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.696µs CommitTime 955.203021ms TotalFlushTime 26m5.341408513s TotalCommitTime 2m35.677662473s 2016-04-07T04:16:51.694-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35509104, 27754552, 287 committed:true 2016-04-07T04:16:51.694-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35509104, 27754552, 287 committed:true) SnapCreateDur 970.374748ms SnapOpenDur 46.998µs 2016-04-07T04:16:51.9-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27880000 QueuedCount 9179 2016-04-07T04:16:55.263-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.788µs CommitTime 301.947074ms TotalFlushTime 26m8.570451405s TotalCommitTime 2m35.979609547s 2016-04-07T04:16:55.263-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35527614, 27763807, 288 committed:true 2016-04-07T04:16:55.263-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35527614, 27763807, 288 committed:true) SnapCreateDur 313.194314ms SnapOpenDur 55.145µs 2016-04-07T04:16:55.288-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:55.301-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27890000 QueuedCount 7643 2016-04-07T04:16:55.311-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:55.333-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:55.356-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:55.38-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:16:56.205-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27900000 QueuedCount 29888 2016-04-07T04:16:57.028-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27910000 QueuedCount 19888 2016-04-07T04:16:57.971-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27920000 QueuedCount 9888 2016-04-07T04:16:58.965-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35607688, 27803844, 0 committed:false 2016-04-07T04:16:59.272-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35607688, 27803844, 0 committed:false) SnapCreateDur 12.877µs SnapOpenDur 307.460118ms 2016-04-07T04:16:59.296-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27930000 QueuedCount 7510 2016-04-07T04:17:03-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27940000 QueuedCount 173 2016-04-07T04:17:03.723-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.843µs CommitTime 703.978353ms TotalFlushTime 26m15.937203818s TotalCommitTime 2m36.6835879s 2016-04-07T04:17:03.723-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35627898, 27813949, 289 committed:true 2016-04-07T04:17:03.724-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35627898, 27813949, 289 committed:true) SnapCreateDur 715.405843ms SnapOpenDur 39.871µs 2016-04-07T04:17:03.749-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:03.774-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:03.799-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:03.825-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:03.915-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:04.803-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27950000 QueuedCount 50000 2016-04-07T04:17:05.789-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27960000 QueuedCount 42313 2016-04-07T04:17:06.656-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27970000 QueuedCount 32313 2016-04-07T04:17:07.515-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27980000 QueuedCount 22313 2016-04-07T04:17:08.416-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 27990000 QueuedCount 12313 2016-04-07T04:17:09.295-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28000000 QueuedCount 2313 2016-04-07T04:17:09.525-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35750190, 27875095, 0 committed:false 2016-04-07T04:17:09.956-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35750190, 27875095, 0 committed:false) SnapCreateDur 16.98µs SnapOpenDur 430.680288ms 2016-04-07T04:17:11.602-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.497µs CommitTime 981.782451ms TotalFlushTime 26m22.282147874s TotalCommitTime 2m37.665370351s 2016-04-07T04:17:11.603-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35764748, 27882374, 290 committed:true 2016-04-07T04:17:11.603-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35764748, 27882374, 290 committed:true) SnapCreateDur 993.446153ms SnapOpenDur 63.288µs 2016-04-07T04:17:11.627-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:11.633-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28010000 QueuedCount 6709 2016-04-07T04:17:11.652-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:11.676-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:11.701-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:11.724-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:12.952-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8714428,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3483054459,"bucket-1:myindex:avg_ts_items_count":24120,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8027798318,"bucket-1:myindex:delete_bytes":339239685,"bucket-1:myindex:disk_size":75333414912,"bucket-1:myindex:flush_queue_size":44009,"bucket-1:myindex:frag_percent":89,"bucket-1:myindex:get_bytes":344719046,"bucket-1:myindex:insert_bytes":2928375870,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":290,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":28524281,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":11475719,"bucket-1:myindex:num_flush_queued":28060730,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":426,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1647813228,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"290 157665370351 7021715232220561231","bucket-1:myindex:timings/storage_del":"7889295 227409027861 1878773776183139385","bucket-1:myindex:timings/storage_get":"28016722 208895702338 764002001933163288","bucket-1:myindex:timings/storage_handle_open":"290 6958170 177982721656","bucket-1:myindex:timings/storage_info":"1433 12703046 315182617612","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"293 195400444 3685157009070496","bucket-1:myindex:timings/storage_meta_set":"290 449826260 107867855753560052","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"55778588 1128989602243 155002582689826923","bucket-1:myindex:timings/storage_snapshot_close":"136 5517641930 236101498514661944","bucket-1:myindex:timings/storage_snapshot_create":"136 51097748275 -8393811522332344275","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":733,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"5437 834409354 149389752035870"} 2016-04-07T04:17:13.562-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28020000 QueuedCount 40730 2016-04-07T04:17:14.38-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28030000 QueuedCount 30730 2016-04-07T04:17:15.241-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28040000 QueuedCount 20730 2016-04-07T04:17:17.844-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28050000 QueuedCount 10730 2016-04-07T04:17:18.925-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28060000 QueuedCount 730 2016-04-07T04:17:19.038-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35865238, 27932619, 0 committed:false 2016-04-07T04:17:19.442-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35865238, 27932619, 0 committed:false) SnapCreateDur 12.969µs SnapOpenDur 403.93839ms 2016-04-07T04:17:20.91-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.74µs CommitTime 742.932501ms TotalFlushTime 26m30.357422617s TotalCommitTime 2m38.408302852s 2016-04-07T04:17:20.91-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35881130, 27940565, 291 committed:true 2016-04-07T04:17:20.911-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35881130, 27940565, 291 committed:true) SnapCreateDur 754.064388ms SnapOpenDur 34.669µs 2016-04-07T04:17:20.936-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:20.959-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28070000 QueuedCount 13716 2016-04-07T04:17:20.961-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:20.986-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:21.011-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:21.036-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:22.054-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28080000 QueuedCount 45062 2016-04-07T04:17:25.833-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28090000 QueuedCount 35062 2016-04-07T04:17:27.018-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28100000 QueuedCount 25062 2016-04-07T04:17:28.173-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28110000 QueuedCount 15062 2016-04-07T04:17:29.329-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28120000 QueuedCount 5062 2016-04-07T04:17:29.899-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 35991886, 27995943, 0 committed:false 2016-04-07T04:17:30.295-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 35991886, 27995943, 0 committed:false) SnapCreateDur 23.851µs SnapOpenDur 396.124176ms 2016-04-07T04:17:30.721-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28130000 QueuedCount 5123 2016-04-07T04:17:32.056-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.742µs CommitTime 814.724162ms TotalFlushTime 26m40.169690195s TotalCommitTime 2m39.223027014s 2016-04-07T04:17:32.056-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36011714, 28005857, 292 committed:true 2016-04-07T04:17:32.056-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36011714, 28005857, 292 committed:true) SnapCreateDur 825.795269ms SnapOpenDur 32.827µs 2016-04-07T04:17:32.08-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:32.103-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:32.125-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:32.149-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:32.174-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:32.551-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28140000 QueuedCount 40638 2016-04-07T04:17:33.486-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28150000 QueuedCount 30638 2016-04-07T04:17:35.412-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28160000 QueuedCount 20638 2016-04-07T04:17:39.467-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28170000 QueuedCount 10638 2016-04-07T04:17:40.329-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28180000 QueuedCount 638 2016-04-07T04:17:40.34-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36101310, 28050655, 0 committed:false 2016-04-07T04:17:40.677-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36101310, 28050655, 0 committed:false) SnapCreateDur 31.176µs SnapOpenDur 337.301484ms 2016-04-07T04:17:41.579-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28190000 QueuedCount 129 2016-04-07T04:17:42.295-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 13.325µs CommitTime 701.83214ms TotalFlushTime 26m49.289453257s TotalCommitTime 2m39.924859154s 2016-04-07T04:17:42.295-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36120022, 28060011, 293 committed:true 2016-04-07T04:17:42.295-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36120022, 28060011, 293 committed:true) SnapCreateDur 712.403746ms SnapOpenDur 48.977µs 2016-04-07T04:17:42.32-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:42.347-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:42.372-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:42.396-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:42.42-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:43.501-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28200000 QueuedCount 47536 2016-04-07T04:17:46.849-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28210000 QueuedCount 37536 2016-04-07T04:17:48.002-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28220000 QueuedCount 27536 2016-04-07T04:17:48.842-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28230000 QueuedCount 17536 2016-04-07T04:17:49.725-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28240000 QueuedCount 7536 2016-04-07T04:17:50.432-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36232992, 28116496, 0 committed:false 2016-04-07T04:17:50.836-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36232992, 28116496, 0 committed:false) SnapCreateDur 17.248µs SnapOpenDur 403.954242ms 2016-04-07T04:17:50.992-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28250000 QueuedCount 8414 2016-04-07T04:17:52.529-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.246µs CommitTime 807.4727ms TotalFlushTime 26m58.199273129s TotalCommitTime 2m40.732331854s 2016-04-07T04:17:52.529-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36254428, 28127214, 294 committed:true 2016-04-07T04:17:52.529-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36254428, 28127214, 294 committed:true) SnapCreateDur 817.646302ms SnapOpenDur 50.594µs 2016-04-07T04:17:52.554-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:52.579-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:52.603-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:52.627-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:52.651-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:52.696-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28260000 QueuedCount 48298 2016-04-07T04:17:53.549-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28270000 QueuedCount 38298 2016-04-07T04:17:54.546-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28280000 QueuedCount 28298 2016-04-07T04:17:55.448-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28290000 QueuedCount 18298 2016-04-07T04:17:56.294-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28300000 QueuedCount 8298 2016-04-07T04:17:57.045-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36352766, 28176383, 0 committed:false 2016-04-07T04:17:57.418-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36352766, 28176383, 0 committed:false) SnapCreateDur 28.875µs SnapOpenDur 372.902541ms 2016-04-07T04:17:57.657-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28310000 QueuedCount 8529 2016-04-07T04:17:59.155-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.531µs CommitTime 799.58658ms TotalFlushTime 27m3.562796179s TotalCommitTime 2m41.531918434s 2016-04-07T04:17:59.155-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36372874, 28186437, 295 committed:true 2016-04-07T04:17:59.155-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36372874, 28186437, 295 committed:true) SnapCreateDur 812.448795ms SnapOpenDur 49.285µs 2016-04-07T04:17:59.18-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:59.205-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:59.229-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:59.253-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:17:59.277-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:01.601-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28320000 QueuedCount 50000 2016-04-07T04:18:02.53-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28330000 QueuedCount 46543 2016-04-07T04:18:03.385-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28340000 QueuedCount 36543 2016-04-07T04:18:04.274-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28350000 QueuedCount 26543 2016-04-07T04:18:05.222-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28360000 QueuedCount 16543 2016-04-07T04:18:06.069-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28370000 QueuedCount 6543 2016-04-07T04:18:06.635-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36487166, 28243583, 0 committed:false 2016-04-07T04:18:07.025-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36487166, 28243583, 0 committed:false) SnapCreateDur 37.093µs SnapOpenDur 390.202136ms 2016-04-07T04:18:07.4-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28380000 QueuedCount 5448 2016-04-07T04:18:08.651-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.806µs CommitTime 771.029351ms TotalFlushTime 27m11.793479324s TotalCommitTime 2m42.302947785s 2016-04-07T04:18:08.652-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36504714, 28252357, 296 committed:true 2016-04-07T04:18:08.652-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36504714, 28252357, 296 committed:true) SnapCreateDur 781.530705ms SnapOpenDur 41.076µs 2016-04-07T04:18:08.675-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:08.698-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:08.721-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:08.744-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:08.769-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:10.426-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28390000 QueuedCount 39308 2016-04-07T04:18:11.411-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28400000 QueuedCount 29308 2016-04-07T04:18:12.952-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":8345850,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3552295064,"bucket-1:myindex:avg_ts_items_count":24938,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8095058070,"bucket-1:myindex:delete_bytes":354851351,"bucket-1:myindex:disk_size":78440648704,"bucket-1:myindex:flush_queue_size":28547,"bucket-1:myindex:frag_percent":89,"bucket-1:myindex:get_bytes":360574264,"bucket-1:myindex:insert_bytes":2966497485,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":296,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":28864729,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":11135271,"bucket-1:myindex:num_flush_queued":28429308,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":438,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1626671889,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"296 162302947785 -7830899207298425939","bucket-1:myindex:timings/storage_del":"8267419 244793879345 -1869349785080469381","bucket-1:myindex:timings/storage_get":"28400762 213540911377 774833378701453947","bucket-1:myindex:timings/storage_handle_open":"296 7094397 181226939053","bucket-1:myindex:timings/storage_info":"1469 13070798 325158958378","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"299 197695026 3686068034117748","bucket-1:myindex:timings/storage_meta_set":"296 452727289 107869287336818229","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"56534836 1157674713362 -6157799598269815580","bucket-1:myindex:timings/storage_snapshot_close":"142 5788348257 248397248521271217","bucket-1:myindex:timings/storage_snapshot_create":"142 53401643306 -7505873184843562280","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":691,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"5580 860624017 154316140636033"} 2016-04-07T04:18:16.503-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28410000 QueuedCount 19308 2016-04-07T04:18:17.329-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28420000 QueuedCount 9308 2016-04-07T04:18:18.144-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36590966, 28295483, 0 committed:false 2016-04-07T04:18:18.501-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36590966, 28295483, 0 committed:false) SnapCreateDur 20.671µs SnapOpenDur 357.607519ms 2016-04-07T04:18:18.536-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28430000 QueuedCount 9291 2016-04-07T04:18:20.085-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.73µs CommitTime 747.940747ms TotalFlushTime 27m22.030896063s TotalCommitTime 2m43.050888532s 2016-04-07T04:18:20.085-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36610632, 28305316, 297 committed:true 2016-04-07T04:18:20.085-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36610632, 28305316, 297 committed:true) SnapCreateDur 758.562737ms SnapOpenDur 34.745µs 2016-04-07T04:18:20.111-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:20.137-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:20.162-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:20.186-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:20.657-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28440000 QueuedCount 50000 2016-04-07T04:18:20.657-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:21.916-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28450000 QueuedCount 50000 2016-04-07T04:18:22.935-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28460000 QueuedCount 40167 2016-04-07T04:18:23.8-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28470000 QueuedCount 30167 2016-04-07T04:18:24.751-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28480000 QueuedCount 20167 2016-04-07T04:18:26.601-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28490000 QueuedCount 10167 2016-04-07T04:18:28.98-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28500000 QueuedCount 167 2016-04-07T04:18:28.99-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36730488, 28365244, 0 committed:false 2016-04-07T04:18:29.407-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36730488, 28365244, 0 committed:false) SnapCreateDur 14.273µs SnapOpenDur 417.031296ms 2016-04-07T04:18:31.159-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.682µs CommitTime 846.717943ms TotalFlushTime 27m31.72714898s TotalCommitTime 2m43.897606475s 2016-04-07T04:18:31.159-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36748474, 28374237, 298 committed:true 2016-04-07T04:18:31.159-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36748474, 28374237, 298 committed:true) SnapCreateDur 857.626517ms SnapOpenDur 40.641µs 2016-04-07T04:18:31.184-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:31.195-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28510000 QueuedCount 9004 2016-04-07T04:18:31.208-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:31.233-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:31.256-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:31.28-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:32.098-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28520000 QueuedCount 39983 2016-04-07T04:18:32.978-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28530000 QueuedCount 29983 2016-04-07T04:18:33.864-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28540000 QueuedCount 19983 2016-04-07T04:18:34.778-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28550000 QueuedCount 9983 2016-04-07T04:18:35.613-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36848194, 28424097, 0 committed:false 2016-04-07T04:18:36.003-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36848194, 28424097, 0 committed:false) SnapCreateDur 16.791µs SnapOpenDur 390.086032ms 2016-04-07T04:18:36.024-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28560000 QueuedCount 742 2016-04-07T04:18:40.965-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.975µs CommitTime 786.55503ms TotalFlushTime 27m40.265833531s TotalCommitTime 2m44.684161505s 2016-04-07T04:18:40.966-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36862478, 28431239, 299 committed:true 2016-04-07T04:18:40.966-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36862478, 28431239, 299 committed:true) SnapCreateDur 798.565203ms SnapOpenDur 60.082µs 2016-04-07T04:18:40.991-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:41.016-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:41.151-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28570000 QueuedCount 27931 2016-04-07T04:18:42.293-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28580000 QueuedCount 17931 2016-04-07T04:18:43.21-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28590000 QueuedCount 7931 2016-04-07T04:18:44.451-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.112µs CommitTime 579.243693ms TotalFlushTime 27m43.116908841s TotalCommitTime 2m45.263405198s 2016-04-07T04:18:44.451-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 36922958, 28461479, 300 committed:true 2016-04-07T04:18:44.451-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 36922958, 28461479, 300 committed:true) SnapCreateDur 590.774492ms SnapOpenDur 43.781µs 2016-04-07T04:18:44.477-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:44.501-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:44.525-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:44.55-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:44.574-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:45.29-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28600000 QueuedCount 46980 2016-04-07T04:18:46.283-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28610000 QueuedCount 36980 2016-04-07T04:18:47.254-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28620000 QueuedCount 26980 2016-04-07T04:18:48.117-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28630000 QueuedCount 16980 2016-04-07T04:18:49.041-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28640000 QueuedCount 6980 2016-04-07T04:18:53.267-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37019500, 28509750, 0 committed:false 2016-04-07T04:18:53.702-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37019500, 28509750, 0 committed:false) SnapCreateDur 20.887µs SnapOpenDur 435.344417ms 2016-04-07T04:18:54.612-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.449µs CommitTime 735.402308ms TotalFlushTime 27m52.020623762s TotalCommitTime 2m45.998807506s 2016-04-07T04:18:54.612-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37024208, 28512104, 301 committed:true 2016-04-07T04:18:54.612-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37024208, 28512104, 301 committed:true) SnapCreateDur 746.935463ms SnapOpenDur 31.5µs 2016-04-07T04:18:54.636-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:54.646-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28650000 QueuedCount 5317 2016-04-07T04:18:54.66-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:54.687-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:54.712-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:54.737-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:18:55.581-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28660000 QueuedCount 49174 2016-04-07T04:18:56.614-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28670000 QueuedCount 39174 2016-04-07T04:18:57.479-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28680000 QueuedCount 29174 2016-04-07T04:18:58.44-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28690000 QueuedCount 19174 2016-04-07T04:18:59.263-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28700000 QueuedCount 9174 2016-04-07T04:18:59.948-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37141972, 28570986, 0 committed:false 2016-04-07T04:19:00.41-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37141972, 28570986, 0 committed:false) SnapCreateDur 8.877µs SnapOpenDur 462.65455ms 2016-04-07T04:19:00.538-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28710000 QueuedCount 9188 2016-04-07T04:19:02.075-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.796µs CommitTime 758.164864ms TotalFlushTime 27m58.154593031s TotalCommitTime 2m46.75697237s 2016-04-07T04:19:02.075-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37161710, 28580855, 302 committed:true 2016-04-07T04:19:02.075-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37161710, 28580855, 302 committed:true) SnapCreateDur 769.219555ms SnapOpenDur 78.668µs 2016-04-07T04:19:02.1-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:02.114-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28720000 QueuedCount 10266 2016-04-07T04:19:02.125-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:02.148-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:02.178-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:02.9-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:03.064-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28730000 QueuedCount 50000 2016-04-07T04:19:04.139-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28740000 QueuedCount 45961 2016-04-07T04:19:05.089-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28750000 QueuedCount 35961 2016-04-07T04:19:05.907-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28760000 QueuedCount 25961 2016-04-07T04:19:06.709-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28770000 QueuedCount 15961 2016-04-07T04:19:07.553-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28780000 QueuedCount 5961 2016-04-07T04:19:08.021-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37293178, 28646589, 0 committed:false 2016-04-07T04:19:08.448-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37293178, 28646589, 0 committed:false) SnapCreateDur 10.451µs SnapOpenDur 427.352367ms 2016-04-07T04:19:08.808-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28790000 QueuedCount 5777 2016-04-07T04:19:10.272-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.39µs CommitTime 1.018544142s TotalFlushTime 28m4.777404411s TotalCommitTime 2m47.775516512s 2016-04-07T04:19:10.273-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37312510, 28656255, 303 committed:true 2016-04-07T04:19:10.273-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37312510, 28656255, 303 committed:true) SnapCreateDur 1.029216568s SnapOpenDur 56.119µs 2016-04-07T04:19:10.297-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:10.321-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:10.345-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:10.369-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:10.393-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:10.793-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28800000 QueuedCount 47659 2016-04-07T04:19:11.671-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28810000 QueuedCount 37659 2016-04-07T04:19:12.533-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28820000 QueuedCount 27659 2016-04-07T04:19:12.952-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":7927499,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3313329087,"bucket-1:myindex:avg_ts_items_count":27848,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8168243432,"bucket-1:myindex:delete_bytes":371815582,"bucket-1:myindex:disk_size":81296973824,"bucket-1:myindex:flush_queue_size":22657,"bucket-1:myindex:frag_percent":89,"bucket-1:myindex:get_bytes":377809008,"bucket-1:myindex:insert_bytes":3007921770,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":303,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":29287220,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":10712780,"bucket-1:myindex:num_flush_queued":28847659,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":451,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1824350957,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"303 167775516512 -3447297815360396008","bucket-1:myindex:timings/storage_del":"8685042 260025978961 -7528366668432694109","bucket-1:myindex:timings/storage_get":"28825003 219234201273 843228313768023083","bucket-1:myindex:timings/storage_handle_open":"303 7299641 188334355357","bucket-1:myindex:timings/storage_info":"1509 13508691 343186096829","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"306 200575866 3687280618196592","bucket-1:myindex:timings/storage_meta_set":"303 456452636 107871285602199322","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"57370083 1189326807398 -707236969614080642","bucket-1:myindex:timings/storage_snapshot_close":"148 6073995422 262196459670249926","bucket-1:myindex:timings/storage_snapshot_create":"148 55891236350 -6466102213538632068","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":646,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"5723 887729434 159626019457496"} 2016-04-07T04:19:17.239-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28830000 QueuedCount 17659 2016-04-07T04:19:18.545-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28840000 QueuedCount 7659 2016-04-07T04:19:19.318-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37414670, 28707335, 0 committed:false 2016-04-07T04:19:19.753-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37414670, 28707335, 0 committed:false) SnapCreateDur 13.734µs SnapOpenDur 434.796075ms 2016-04-07T04:19:19.943-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28850000 QueuedCount 8010 2016-04-07T04:19:21.458-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.462µs CommitTime 779.418692ms TotalFlushTime 28m14.655626973s TotalCommitTime 2m48.554935204s 2016-04-07T04:19:21.458-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37435034, 28717517, 304 committed:true 2016-04-07T04:19:21.459-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37435034, 28717517, 304 committed:true) SnapCreateDur 790.548526ms SnapOpenDur 46.576µs 2016-04-07T04:19:21.483-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:21.508-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:21.533-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:21.556-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:21.578-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:21.634-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28860000 QueuedCount 43638 2016-04-07T04:19:22.531-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28870000 QueuedCount 33638 2016-04-07T04:19:23.421-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28880000 QueuedCount 23638 2016-04-07T04:19:24.392-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28890000 QueuedCount 13638 2016-04-07T04:19:25.227-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28900000 QueuedCount 3638 2016-04-07T04:19:25.561-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37524894, 28762447, 0 committed:false 2016-04-07T04:19:25.926-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37524894, 28762447, 0 committed:false) SnapCreateDur 12.399µs SnapOpenDur 364.756315ms 2016-04-07T04:19:26.622-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28910000 QueuedCount 3352 2016-04-07T04:19:27.924-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.344µs CommitTime 991.618367ms TotalFlushTime 28m19.67261574s TotalCommitTime 2m49.546553571s 2016-04-07T04:19:27.924-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37544018, 28772009, 305 committed:true 2016-04-07T04:19:27.924-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37544018, 28772009, 305 committed:true) SnapCreateDur 1.003167821s SnapOpenDur 45.406µs 2016-04-07T04:19:27.949-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:27.974-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:27.998-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:28.022-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:31.512-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:32.434-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28920000 QueuedCount 50000 2016-04-07T04:19:33.39-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28930000 QueuedCount 45466 2016-04-07T04:19:34.321-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28940000 QueuedCount 35466 2016-04-07T04:19:35.12-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28950000 QueuedCount 25466 2016-04-07T04:19:35.941-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28960000 QueuedCount 15466 2016-04-07T04:19:36.736-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28970000 QueuedCount 5466 2016-04-07T04:19:38.03-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.337µs CommitTime 834.627816ms TotalFlushTime 28m28.850988845s TotalCommitTime 2m50.381181387s 2016-04-07T04:19:38.03-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37666280, 28833140, 306 committed:true 2016-04-07T04:19:38.03-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37666280, 28833140, 306 committed:true) SnapCreateDur 845.981017ms SnapOpenDur 53.841µs 2016-04-07T04:19:38.528-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28980000 QueuedCount 6178 2016-04-07T04:19:40.044-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.07µs CommitTime 278.193466ms TotalFlushTime 28m30.547757288s TotalCommitTime 2m50.659374853s 2016-04-07T04:19:40.044-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37687306, 28843653, 307 committed:true 2016-04-07T04:19:40.044-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37687306, 28843653, 307 committed:true) SnapCreateDur 289.192271ms SnapOpenDur 53.934µs 2016-04-07T04:19:40.069-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:40.093-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:40.117-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:40.141-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:40.166-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:42.145-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 28990000 QueuedCount 50000 2016-04-07T04:19:43.514-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29000000 QueuedCount 40307 2016-04-07T04:19:44.65-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29010000 QueuedCount 30307 2016-04-07T04:19:48.221-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29020000 QueuedCount 20307 2016-04-07T04:19:49.03-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29030000 QueuedCount 10307 2016-04-07T04:19:49.827-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29040000 QueuedCount 307 2016-04-07T04:19:49.842-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37793738, 28896869, 0 committed:false 2016-04-07T04:19:50.287-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37793738, 28896869, 0 committed:false) SnapCreateDur 17.624µs SnapOpenDur 445.089609ms 2016-04-07T04:19:51.866-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.167µs CommitTime 784.304712ms TotalFlushTime 28m41.040196689s TotalCommitTime 2m51.443679565s 2016-04-07T04:19:51.866-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37809840, 28904920, 308 committed:true 2016-04-07T04:19:51.866-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37809840, 28904920, 308 committed:true) SnapCreateDur 794.963118ms SnapOpenDur 37.412µs 2016-04-07T04:19:51.89-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:51.915-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:51.938-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:51.963-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:51.986-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:19:55.462-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29050000 QueuedCount 44854 2016-04-07T04:19:57.275-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29060000 QueuedCount 34854 2016-04-07T04:19:58.18-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29070000 QueuedCount 24854 2016-04-07T04:19:59.076-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29080000 QueuedCount 14854 2016-04-07T04:19:59.897-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29090000 QueuedCount 4854 2016-04-07T04:20:00.681-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37901188, 28950594, 0 committed:false 2016-04-07T04:20:01.046-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37901188, 28950594, 0 committed:false) SnapCreateDur 10.518µs SnapOpenDur 365.663223ms 2016-04-07T04:20:01.415-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29100000 QueuedCount 3157 2016-04-07T04:20:02.622-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.686µs CommitTime 861.595812ms TotalFlushTime 28m50.473605727s TotalCommitTime 2m52.305275377s 2016-04-07T04:20:02.622-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 37917536, 28958768, 309 committed:true 2016-04-07T04:20:02.622-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 37917536, 28958768, 309 committed:true) SnapCreateDur 872.253785ms SnapOpenDur 40.632µs 2016-04-07T04:20:02.647-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:02.671-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:02.695-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:02.719-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:02.744-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:04.252-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29110000 QueuedCount 49736 2016-04-07T04:20:08.994-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29120000 QueuedCount 39736 2016-04-07T04:20:09.949-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29130000 QueuedCount 29736 2016-04-07T04:20:10.792-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29140000 QueuedCount 19736 2016-04-07T04:20:11.66-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29150000 QueuedCount 9736 2016-04-07T04:20:12.489-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38028900, 29014450, 0 committed:false 2016-04-07T04:20:12.892-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38028900, 29014450, 0 committed:false) SnapCreateDur 13.111µs SnapOpenDur 402.987976ms 2016-04-07T04:20:12.919-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29160000 QueuedCount 4885 2016-04-07T04:20:12.952-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":7610273,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":7230194447,"bucket-1:myindex:avg_ts_items_count":39193,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8228363220,"bucket-1:myindex:delete_bytes":385882688,"bucket-1:myindex:disk_size":84521431040,"bucket-1:myindex:flush_queue_size":4508,"bucket-1:myindex:frag_percent":90,"bucket-1:myindex:get_bytes":392102079,"bucket-1:myindex:insert_bytes":3042271680,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":309,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":29669500,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":10330500,"bucket-1:myindex:num_flush_queued":29164885,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":462,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":10269891030,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"309 172305275377 274979088137706845","bucket-1:myindex:timings/storage_del":"9015079 278505009640 -2550049163568230140","bucket-1:myindex:timings/storage_get":"29160378 223789472458 898593696598476030","bucket-1:myindex:timings/storage_handle_open":"309 7424589 190974678079","bucket-1:myindex:timings/storage_info":"1543 13798192 350416897144","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"312 202815964 3688122863362658","bucket-1:myindex:timings/storage_meta_set":"309 459432663 107872838400574013","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"58030156 1218955883191 -168232611928190243","bucket-1:myindex:timings/storage_snapshot_close":"152 6255212564 270530149664695960","bucket-1:myindex:timings/storage_snapshot_create":"153 57902944048 -5650983054843567496","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":610,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"5867 914931663 164963511109441"} 2016-04-07T04:20:14.181-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.782µs CommitTime 775.60908ms TotalFlushTime 29m0.750928566s TotalCommitTime 2m53.080884457s 2016-04-07T04:20:14.182-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38039026, 29019513, 310 committed:true 2016-04-07T04:20:14.182-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38039026, 29019513, 310 committed:true) SnapCreateDur 786.327207ms SnapOpenDur 55.095µs 2016-04-07T04:20:14.204-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:14.227-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:14.251-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:14.275-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:14.301-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:14.619-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29170000 QueuedCount 48397 2016-04-07T04:20:15.482-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29180000 QueuedCount 38397 2016-04-07T04:20:16.994-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29190000 QueuedCount 28397 2016-04-07T04:20:17.994-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29200000 QueuedCount 18397 2016-04-07T04:20:18.803-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29210000 QueuedCount 8397 2016-04-07T04:20:19.503-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38144434, 29072217, 0 committed:false 2016-04-07T04:20:19.891-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38144434, 29072217, 0 committed:false) SnapCreateDur 13.02µs SnapOpenDur 388.193348ms 2016-04-07T04:20:20.03-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29220000 QueuedCount 9144 2016-04-07T04:20:22.537-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.679µs CommitTime 879.90969ms TotalFlushTime 29m7.741119753s TotalCommitTime 2m53.960794147s 2016-04-07T04:20:22.537-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38165622, 29082811, 311 committed:true 2016-04-07T04:20:22.537-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38165622, 29082811, 311 committed:true) SnapCreateDur 890.888633ms SnapOpenDur 38.65µs 2016-04-07T04:20:22.562-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:22.579-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29230000 QueuedCount 10024 2016-04-07T04:20:22.587-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:22.611-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:22.635-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:22.659-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:23.557-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29240000 QueuedCount 44134 2016-04-07T04:20:24.427-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29250000 QueuedCount 34134 2016-04-07T04:20:25.291-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29260000 QueuedCount 24134 2016-04-07T04:20:26.176-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29270000 QueuedCount 14134 2016-04-07T04:20:26.99-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29280000 QueuedCount 4134 2016-04-07T04:20:27.309-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38273930, 29136965, 0 committed:false 2016-04-07T04:20:27.686-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38273930, 29136965, 0 committed:false) SnapCreateDur 11.691µs SnapOpenDur 377.015881ms 2016-04-07T04:20:28.141-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29290000 QueuedCount 3133 2016-04-07T04:20:29.307-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.968µs CommitTime 934.267053ms TotalFlushTime 29m13.100849331s TotalCommitTime 2m54.8950612s 2016-04-07T04:20:29.307-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38291650, 29145825, 312 committed:true 2016-04-07T04:20:29.307-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38291650, 29145825, 312 committed:true) SnapCreateDur 945.87071ms SnapOpenDur 57.432µs 2016-04-07T04:20:29.333-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:29.357-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:29.381-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:29.406-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:29.431-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:34.001-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29300000 QueuedCount 44069 2016-04-07T04:20:34.863-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29310000 QueuedCount 34069 2016-04-07T04:20:35.715-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29320000 QueuedCount 24069 2016-04-07T04:20:36.634-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29330000 QueuedCount 14069 2016-04-07T04:20:38.36-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29340000 QueuedCount 4069 2016-04-07T04:20:41.654-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38391856, 29195928, 0 committed:false 2016-04-07T04:20:42.012-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38391856, 29195928, 0 committed:false) SnapCreateDur 11.694µs SnapOpenDur 357.272153ms 2016-04-07T04:20:42.498-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29350000 QueuedCount 1392 2016-04-07T04:20:43.494-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.782µs CommitTime 867.337185ms TotalFlushTime 29m25.969561406s TotalCommitTime 2m55.762398385s 2016-04-07T04:20:43.494-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38406268, 29203134, 313 committed:true 2016-04-07T04:20:43.494-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38406268, 29203134, 313 committed:true) SnapCreateDur 878.087929ms SnapOpenDur 36.594µs 2016-04-07T04:20:43.518-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:43.542-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:43.566-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:43.591-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:43.616-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:44.323-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29360000 QueuedCount 46621 2016-04-07T04:20:45.218-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29370000 QueuedCount 36621 2016-04-07T04:20:46.005-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29380000 QueuedCount 26621 2016-04-07T04:20:46.763-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29390000 QueuedCount 16621 2016-04-07T04:20:48.522-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29400000 QueuedCount 6621 2016-04-07T04:20:49.097-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38515028, 29257514, 0 committed:false 2016-04-07T04:20:49.489-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38515028, 29257514, 0 committed:false) SnapCreateDur 16.696µs SnapOpenDur 391.675557ms 2016-04-07T04:20:49.748-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29410000 QueuedCount 5367 2016-04-07T04:20:51.015-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.454µs CommitTime 820.725424ms TotalFlushTime 29m32.175934398s TotalCommitTime 2m56.583123809s 2016-04-07T04:20:51.015-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38532280, 29266140, 314 committed:true 2016-04-07T04:20:51.015-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38532280, 29266140, 314 committed:true) SnapCreateDur 831.33097ms SnapOpenDur 42.718µs 2016-04-07T04:20:51.039-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:51.063-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:51.085-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:51.107-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:51.131-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:51.453-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29420000 QueuedCount 34203 2016-04-07T04:20:52.301-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29430000 QueuedCount 24203 2016-04-07T04:20:53.224-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29440000 QueuedCount 14203 2016-04-07T04:20:54.056-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29450000 QueuedCount 4203 2016-04-07T04:20:54.415-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38608784, 29304392, 0 committed:false 2016-04-07T04:20:54.731-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38608784, 29304392, 0 committed:false) SnapCreateDur 8.272µs SnapOpenDur 315.698875ms 2016-04-07T04:20:55.237-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29460000 QueuedCount 3582 2016-04-07T04:20:56.248-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.337µs CommitTime 672.415165ms TotalFlushTime 29m36.344273283s TotalCommitTime 2m57.255538974s 2016-04-07T04:20:56.248-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38627214, 29313607, 315 committed:true 2016-04-07T04:20:56.248-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38627214, 29313607, 315 committed:true) SnapCreateDur 682.692793ms SnapOpenDur 52.152µs 2016-04-07T04:20:56.273-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:56.298-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:56.323-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:56.347-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:56.561-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:20:56.836-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29470000 QueuedCount 50000 2016-04-07T04:20:57.749-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29480000 QueuedCount 47455 2016-04-07T04:20:58.637-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29490000 QueuedCount 37455 2016-04-07T04:20:59.603-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29500000 QueuedCount 27455 2016-04-07T04:21:00.405-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29510000 QueuedCount 17455 2016-04-07T04:21:01.179-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29520000 QueuedCount 7455 2016-04-07T04:21:01.735-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38752978, 29376489, 0 committed:false 2016-04-07T04:21:02.183-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38752978, 29376489, 0 committed:false) SnapCreateDur 12.313µs SnapOpenDur 448.395939ms 2016-04-07T04:21:02.444-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29530000 QueuedCount 6539 2016-04-07T04:21:03.811-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 10.061µs CommitTime 812.556278ms TotalFlushTime 29m42.526652703s TotalCommitTime 2m58.068095252s 2016-04-07T04:21:03.811-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38770888, 29385444, 316 committed:true 2016-04-07T04:21:03.811-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38770888, 29385444, 316 committed:true) SnapCreateDur 823.27394ms SnapOpenDur 46.168µs 2016-04-07T04:21:03.835-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:03.859-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:03.883-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:03.907-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:03.932-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:04.112-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29540000 QueuedCount 49012 2016-04-07T04:21:04.919-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29550000 QueuedCount 39012 2016-04-07T04:21:05.842-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29560000 QueuedCount 29012 2016-04-07T04:21:08.849-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29570000 QueuedCount 19012 2016-04-07T04:21:11.292-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29580000 QueuedCount 9012 2016-04-07T04:21:12.107-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38874122, 29437061, 0 committed:false 2016-04-07T04:21:12.594-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38874122, 29437061, 0 committed:false) SnapCreateDur 15.032µs SnapOpenDur 486.576641ms 2016-04-07T04:21:12.635-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29590000 QueuedCount 7923 2016-04-07T04:21:12.953-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":7177235,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":5858497412,"bucket-1:myindex:avg_ts_items_count":39035,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8310627515,"bucket-1:myindex:delete_bytes":405004831,"bucket-1:myindex:disk_size":88127975424,"bucket-1:myindex:flush_queue_size":4133,"bucket-1:myindex:frag_percent":90,"bucket-1:myindex:get_bytes":411524491,"bucket-1:myindex:insert_bytes":3088965242,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":316,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":30127122,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":9872878,"bucket-1:myindex:num_flush_queued":29597923,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":476,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":8783071100,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"316 178068095252 5061898395116248664","bucket-1:myindex:timings/storage_del":"9441754 291019338173 64391342007017733","bucket-1:myindex:timings/storage_get":"29593791 228809009599 929572772108335735","bucket-1:myindex:timings/storage_handle_open":"316 7592403 195352606083","bucket-1:myindex:timings/storage_info":"1585 14151540 358225572048","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"319 205702837 3689375521267681","bucket-1:myindex:timings/storage_meta_set":"316 463104091 107874806107762513","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"58883508 1251955241282 7084987649834698678","bucket-1:myindex:timings/storage_snapshot_close":"159 6594398583 287081742321398665","bucket-1:myindex:timings/storage_snapshot_create":"160 60667261621 -4540018108921653373","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":561,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"6011 941841832 170095527563558"} 2016-04-07T04:21:14.1-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.7µs CommitTime 790.174379ms TotalFlushTime 29m51.446323525s TotalCommitTime 2m58.858269631s 2016-04-07T04:21:14.1-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38891636, 29445818, 317 committed:true 2016-04-07T04:21:14.1-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38891636, 29445818, 317 committed:true) SnapCreateDur 801.212826ms SnapOpenDur 43.036µs 2016-04-07T04:21:14.126-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:14.151-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:14.173-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:14.196-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:14.221-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:14.271-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29600000 QueuedCount 44145 2016-04-07T04:21:15.154-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29610000 QueuedCount 34145 2016-04-07T04:21:20.768-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29620000 QueuedCount 24145 2016-04-07T04:21:21.615-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29630000 QueuedCount 14145 2016-04-07T04:21:22.473-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29640000 QueuedCount 4145 2016-04-07T04:21:22.836-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 38982622, 29491311, 0 committed:false 2016-04-07T04:21:23.255-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 38982622, 29491311, 0 committed:false) SnapCreateDur 11.175µs SnapOpenDur 418.933852ms 2016-04-07T04:21:23.753-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29650000 QueuedCount 3343 2016-04-07T04:21:25.127-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.681µs CommitTime 942.842674ms TotalFlushTime 30m1.023736172s TotalCommitTime 2m59.801112305s 2016-04-07T04:21:25.127-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39000730, 29500365, 318 committed:true 2016-04-07T04:21:25.127-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39000730, 29500365, 318 committed:true) SnapCreateDur 955.506336ms SnapOpenDur 46.626µs 2016-04-07T04:21:25.153-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:25.178-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:25.203-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:25.229-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:25.254-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:25.689-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29660000 QueuedCount 47023 2016-04-07T04:21:26.6-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29670000 QueuedCount 37023 2016-04-07T04:21:27.612-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29680000 QueuedCount 27023 2016-04-07T04:21:28.435-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29690000 QueuedCount 17023 2016-04-07T04:21:29.982-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29700000 QueuedCount 7023 2016-04-07T04:21:30.551-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39106504, 29553252, 0 committed:false 2016-04-07T04:21:30.997-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39106504, 29553252, 0 committed:false) SnapCreateDur 17.741µs SnapOpenDur 446.055939ms 2016-04-07T04:21:31.257-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29710000 QueuedCount 2368 2016-04-07T04:21:32.256-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.042µs CommitTime 764.151247ms TotalFlushTime 30m6.849467396s TotalCommitTime 3m0.565263552s 2016-04-07T04:21:32.258-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39117032, 29558516, 319 committed:true 2016-04-07T04:21:32.258-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39117032, 29558516, 319 committed:true) SnapCreateDur 776.957776ms SnapOpenDur 54.4µs 2016-04-07T04:21:32.282-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:32.308-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:32.334-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:32.36-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:32.424-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:32.982-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29720000 QueuedCount 50000 2016-04-07T04:21:33.913-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29730000 QueuedCount 44795 2016-04-07T04:21:34.771-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29740000 QueuedCount 34795 2016-04-07T04:21:35.631-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29750000 QueuedCount 24795 2016-04-07T04:21:36.556-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29760000 QueuedCount 14795 2016-04-07T04:21:37.395-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29770000 QueuedCount 4795 2016-04-07T04:21:37.76-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39239908, 29619954, 0 committed:false 2016-04-07T04:21:38.223-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39239908, 29619954, 0 committed:false) SnapCreateDur 9.984µs SnapOpenDur 462.76515ms 2016-04-07T04:21:38.703-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29780000 QueuedCount 4231 2016-04-07T04:21:40.042-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.928µs CommitTime 982.753578ms TotalFlushTime 30m13.078203904s TotalCommitTime 3m1.54801713s 2016-04-07T04:21:40.042-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39258436, 29629218, 320 committed:true 2016-04-07T04:21:40.043-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39258436, 29629218, 320 committed:true) SnapCreateDur 994.322312ms SnapOpenDur 63.737µs 2016-04-07T04:21:40.067-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:40.092-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:40.118-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:40.142-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:40.167-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:40.707-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29790000 QueuedCount 49627 2016-04-07T04:21:41.571-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29800000 QueuedCount 39627 2016-04-07T04:21:42.41-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29810000 QueuedCount 29627 2016-04-07T04:21:43.234-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29820000 QueuedCount 19627 2016-04-07T04:21:44.073-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29830000 QueuedCount 9627 2016-04-07T04:21:44.871-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39367558, 29683779, 0 committed:false 2016-04-07T04:21:45.257-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39367558, 29683779, 0 committed:false) SnapCreateDur 11.796µs SnapOpenDur 385.70372ms 2016-04-07T04:21:45.286-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29840000 QueuedCount 8119 2016-04-07T04:21:50.99-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.623µs CommitTime 866.889277ms TotalFlushTime 30m22.674320659s TotalCommitTime 3m2.414906407s 2016-04-07T04:21:50.99-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39384242, 29692121, 321 committed:true 2016-04-07T04:21:50.99-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39384242, 29692121, 321 committed:true) SnapCreateDur 878.046874ms SnapOpenDur 51.967µs 2016-04-07T04:21:52.559-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29850000 QueuedCount 5624 2016-04-07T04:21:54.324-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.046µs CommitTime 189.069633ms TotalFlushTime 30m25.780469142s TotalCommitTime 3m2.60397604s 2016-04-07T04:21:54.324-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39399026, 29699513, 322 committed:true 2016-04-07T04:21:54.324-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39399026, 29699513, 322 committed:true) SnapCreateDur 200.079105ms SnapOpenDur 32.969µs 2016-04-07T04:21:54.348-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:54.371-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:54.394-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:54.419-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:54.444-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:21:58.772-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29860000 QueuedCount 46868 2016-04-07T04:21:59.639-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29870000 QueuedCount 36868 2016-04-07T04:22:00.532-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29880000 QueuedCount 26868 2016-04-07T04:22:02.673-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29890000 QueuedCount 16868 2016-04-07T04:22:04.983-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29900000 QueuedCount 6868 2016-04-07T04:22:05.588-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39499902, 29749951, 0 committed:false 2016-04-07T04:22:06.017-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39499902, 29749951, 0 committed:false) SnapCreateDur 12.59µs SnapOpenDur 429.529409ms 2016-04-07T04:22:06.326-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29910000 QueuedCount 3590 2016-04-07T04:22:07.483-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.841µs CommitTime 741.839606ms TotalFlushTime 30m37.674655991s TotalCommitTime 3m3.345815646s 2016-04-07T04:22:07.483-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39513130, 29756565, 323 committed:true 2016-04-07T04:22:07.484-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39513130, 29756565, 323 committed:true) SnapCreateDur 753.005015ms SnapOpenDur 52.594µs 2016-04-07T04:22:07.507-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:07.532-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:07.556-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:07.581-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:07.606-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:08.076-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29920000 QueuedCount 49439 2016-04-07T04:22:08.973-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29930000 QueuedCount 39439 2016-04-07T04:22:09.819-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29940000 QueuedCount 29439 2016-04-07T04:22:10.692-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29950000 QueuedCount 19439 2016-04-07T04:22:11.617-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29960000 QueuedCount 9439 2016-04-07T04:22:12.326-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39623054, 29811527, 0 committed:false 2016-04-07T04:22:12.772-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39623054, 29811527, 0 committed:false) SnapCreateDur 25.188µs SnapOpenDur 445.070137ms 2016-04-07T04:22:12.923-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29970000 QueuedCount 9422 2016-04-07T04:22:12.953-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":6795736,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":4991028813,"bucket-1:myindex:avg_ts_items_count":38012,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8377557561,"bucket-1:myindex:delete_bytes":420424115,"bucket-1:myindex:disk_size":91296337920,"bucket-1:myindex:flush_queue_size":8530,"bucket-1:myindex:frag_percent":90,"bucket-1:myindex:get_bytes":427189993,"bucket-1:myindex:insert_bytes":3126616920,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":323,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":30522238,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":9477762,"bucket-1:myindex:num_flush_queued":29979422,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":489,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":5287999897,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"323 183345815646 -8984215752355767288","bucket-1:myindex:timings/storage_del":"9812957 304904034284 2492401891064099338","bucket-1:myindex:timings/storage_get":"29970893 233807919526 996662547428819276","bucket-1:myindex:timings/storage_handle_open":"323 7737626 198404339962","bucket-1:myindex:timings/storage_info":"1625 14497681 367625759985","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"326 208586493 3690569651891803","bucket-1:myindex:timings/storage_meta_set":"323 466580594 107876538010101372","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"59625913 1284318519066 7050848097721115764","bucket-1:myindex:timings/storage_snapshot_close":"165 6900669112 302841502581730290","bucket-1:myindex:timings/storage_snapshot_create":"166 63254728846 -3420554380836429506","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":518,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"6155 968672811 175192667557577"} 2016-04-07T04:22:14.742-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.445µs CommitTime 815.145478ms TotalFlushTime 30m43.572082661s TotalCommitTime 3m4.160961124s 2016-04-07T04:22:14.742-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39642694, 29821347, 324 committed:true 2016-04-07T04:22:14.742-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39642694, 29821347, 324 committed:true) SnapCreateDur 825.583042ms SnapOpenDur 39.377µs 2016-04-07T04:22:14.767-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:14.776-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29980000 QueuedCount 8876 2016-04-07T04:22:14.791-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:14.813-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:14.835-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:14.86-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:15.728-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 29990000 QueuedCount 32653 2016-04-07T04:22:16.607-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30000000 QueuedCount 22653 2016-04-07T04:22:17.485-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30010000 QueuedCount 12653 2016-04-07T04:22:18.435-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30020000 QueuedCount 2653 2016-04-07T04:22:18.575-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39727654, 29863827, 0 committed:false 2016-04-07T04:22:18.981-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39727654, 29863827, 0 committed:false) SnapCreateDur 11.355µs SnapOpenDur 406.82064ms 2016-04-07T04:22:19.871-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30030000 QueuedCount 4373 2016-04-07T04:22:21.002-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.361µs CommitTime 723.361172ms TotalFlushTime 30m48.613666571s TotalCommitTime 3m4.884322296s 2016-04-07T04:22:21.002-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39750788, 29875394, 325 committed:true 2016-04-07T04:22:21.002-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39750788, 29875394, 325 committed:true) SnapCreateDur 734.601642ms SnapOpenDur 35.359µs 2016-04-07T04:22:21.028-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:21.054-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:21.078-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:21.103-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:21.198-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:21.61-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30040000 QueuedCount 50000 2016-04-07T04:22:22.561-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30050000 QueuedCount 46671 2016-04-07T04:22:23.443-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30060000 QueuedCount 36671 2016-04-07T04:22:28.306-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30070000 QueuedCount 26671 2016-04-07T04:22:29.164-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30080000 QueuedCount 16671 2016-04-07T04:22:30.035-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30090000 QueuedCount 6671 2016-04-07T04:22:30.765-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39873470, 29936735, 0 committed:false 2016-04-07T04:22:31.241-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39873470, 29936735, 0 committed:false) SnapCreateDur 18.594µs SnapOpenDur 475.700645ms 2016-04-07T04:22:31.511-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30100000 QueuedCount 5025 2016-04-07T04:22:32.912-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.354µs CommitTime 915.548021ms TotalFlushTime 30m59.016850457s TotalCommitTime 3m5.799870317s 2016-04-07T04:22:32.912-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39889942, 29944971, 326 committed:true 2016-04-07T04:22:32.912-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39889942, 29944971, 326 committed:true) SnapCreateDur 925.900491ms SnapOpenDur 39.533µs 2016-04-07T04:22:32.937-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:32.961-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:32.984-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:33.008-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:33.033-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:36.921-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30110000 QueuedCount 45611 2016-04-07T04:22:37.781-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30120000 QueuedCount 35611 2016-04-07T04:22:38.682-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30130000 QueuedCount 25611 2016-04-07T04:22:39.629-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30140000 QueuedCount 15611 2016-04-07T04:22:40.646-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30150000 QueuedCount 5611 2016-04-07T04:22:41.138-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 39989436, 29994718, 0 committed:false 2016-04-07T04:22:41.57-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 39989436, 29994718, 0 committed:false) SnapCreateDur 12.7µs SnapOpenDur 432.25812ms 2016-04-07T04:22:41.972-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30160000 QueuedCount 5135 2016-04-07T04:22:43.254-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.509µs CommitTime 898.762936ms TotalFlushTime 31m7.942874929s TotalCommitTime 3m6.698633253s 2016-04-07T04:22:43.254-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40008162, 30004081, 327 committed:true 2016-04-07T04:22:43.254-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40008162, 30004081, 327 committed:true) SnapCreateDur 909.370386ms SnapOpenDur 46.925µs 2016-04-07T04:22:43.278-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:43.302-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:43.326-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:43.351-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:43.374-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:43.711-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30170000 QueuedCount 44944 2016-04-07T04:22:45.991-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30180000 QueuedCount 34944 2016-04-07T04:22:46.875-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30190000 QueuedCount 24944 2016-04-07T04:22:47.715-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30200000 QueuedCount 14944 2016-04-07T04:22:48.669-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30210000 QueuedCount 4944 2016-04-07T04:22:49.208-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40106150, 30053075, 0 committed:false 2016-04-07T04:22:49.579-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40106150, 30053075, 0 committed:false) SnapCreateDur 12.03µs SnapOpenDur 371.448699ms 2016-04-07T04:22:50.094-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30220000 QueuedCount 1883 2016-04-07T04:22:51.003-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.527µs CommitTime 759.639544ms TotalFlushTime 31m14.46536637s TotalCommitTime 3m7.458272797s 2016-04-07T04:22:51.003-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40119808, 30059904, 328 committed:true 2016-04-07T04:22:51.003-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40119808, 30059904, 328 committed:true) SnapCreateDur 770.814916ms SnapOpenDur 30.841µs 2016-04-07T04:22:51.028-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:51.053-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:51.078-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:51.103-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:51.193-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:51.767-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30230000 QueuedCount 50000 2016-04-07T04:22:52.706-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30240000 QueuedCount 42874 2016-04-07T04:22:53.673-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30250000 QueuedCount 32874 2016-04-07T04:22:54.68-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30260000 QueuedCount 22874 2016-04-07T04:22:55.568-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30270000 QueuedCount 12874 2016-04-07T04:22:56.421-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30280000 QueuedCount 2874 2016-04-07T04:22:56.672-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40239852, 30119926, 0 committed:false 2016-04-07T04:22:57.094-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40239852, 30119926, 0 committed:false) SnapCreateDur 19.623µs SnapOpenDur 422.399402ms 2016-04-07T04:22:58.536-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.628µs CommitTime 917.782402ms TotalFlushTime 31m20.540277025s TotalCommitTime 3m8.376055199s 2016-04-07T04:22:58.536-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40249300, 30124650, 329 committed:true 2016-04-07T04:22:58.536-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40249300, 30124650, 329 committed:true) SnapCreateDur 929.034127ms SnapOpenDur 33.738µs 2016-04-07T04:22:58.559-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:58.582-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:58.605-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:58.631-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:58.655-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:22:58.721-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30290000 QueuedCount 48806 2016-04-07T04:22:59.683-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30300000 QueuedCount 38806 2016-04-07T04:23:00.743-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30310000 QueuedCount 28806 2016-04-07T04:23:04.958-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30320000 QueuedCount 18806 2016-04-07T04:23:06.098-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30330000 QueuedCount 8806 2016-04-07T04:23:06.93-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40349840, 30174920, 0 committed:false 2016-04-07T04:23:07.384-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40349840, 30174920, 0 committed:false) SnapCreateDur 11.353µs SnapOpenDur 453.191095ms 2016-04-07T04:23:07.529-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30340000 QueuedCount 10626 2016-04-07T04:23:08.4-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30350000 QueuedCount 626 2016-04-07T04:23:09.298-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.916µs CommitTime 766.917234ms TotalFlushTime 31m29.981533542s TotalCommitTime 3m9.142972433s 2016-04-07T04:23:09.298-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40373146, 30186573, 330 committed:true 2016-04-07T04:23:09.299-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40373146, 30186573, 330 committed:true) SnapCreateDur 778.503966ms SnapOpenDur 45.916µs 2016-04-07T04:23:09.325-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:09.35-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:09.375-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:09.399-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:09.423-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:10.17-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30360000 QueuedCount 45088 2016-04-07T04:23:12.953-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":6370070,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3936703457,"bucket-1:myindex:avg_ts_items_count":24470,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8454334549,"bucket-1:myindex:delete_bytes":438199369,"bucket-1:myindex:disk_size":94910345216,"bucket-1:myindex:flush_queue_size":44712,"bucket-1:myindex:frag_percent":91,"bucket-1:myindex:get_bytes":445256572,"bucket-1:myindex:insert_bytes":3170021672,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":330,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":30974903,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":9025097,"bucket-1:myindex:num_flush_queued":30405088,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":502,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1914889277,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"330 189142972433 -4142960404974720387","bucket-1:myindex:timings/storage_del":"10196162 320744725831 -1489796152131652827","bucket-1:myindex:timings/storage_get":"30360377 238854122674 1049863602616449914","bucket-1:myindex:timings/storage_handle_open":"330 7886357 201869753657","bucket-1:myindex:timings/storage_info":"1665 14873733 379259049291","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"333 211220333 3691566951306615","bucket-1:myindex:timings/storage_meta_set":"330 470094764 107878358473480730","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"60392322 1311450411788 -3447175786745970664","bucket-1:myindex:timings/storage_snapshot_close":"172 7252571270 320617992640851514","bucket-1:myindex:timings/storage_snapshot_create":"172 65814635816 -2321780411065837514","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":470,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"6299 995667600 180402026305638"} 2016-04-07T04:23:16.664-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30370000 QueuedCount 35088 2016-04-07T04:23:17.807-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30380000 QueuedCount 25088 2016-04-07T04:23:22.134-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30390000 QueuedCount 15088 2016-04-07T04:23:23.057-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30400000 QueuedCount 5088 2016-04-07T04:23:23.545-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40480298, 30240149, 0 committed:false 2016-04-07T04:23:23.987-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40480298, 30240149, 0 committed:false) SnapCreateDur 13.923µs SnapOpenDur 441.491553ms 2016-04-07T04:23:24.384-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30410000 QueuedCount 5608 2016-04-07T04:23:25.768-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.917µs CommitTime 883.560465ms TotalFlushTime 31m45.024992552s TotalCommitTime 3m10.026532898s 2016-04-07T04:23:25.768-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40501026, 30250513, 331 committed:true 2016-04-07T04:23:25.768-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40501026, 30250513, 331 committed:true) SnapCreateDur 895.025644ms SnapOpenDur 41.527µs 2016-04-07T04:23:25.793-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:25.819-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:25.845-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:25.871-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:25.896-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:26.336-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30420000 QueuedCount 50000 2016-04-07T04:23:27.323-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30430000 QueuedCount 42535 2016-04-07T04:23:28.251-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30440000 QueuedCount 32535 2016-04-07T04:23:29.162-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30450000 QueuedCount 22535 2016-04-07T04:23:30.106-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30460000 QueuedCount 12535 2016-04-07T04:23:31.077-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30470000 QueuedCount 2535 2016-04-07T04:23:31.218-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40613190, 30306595, 0 committed:false 2016-04-07T04:23:31.703-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40613190, 30306595, 0 committed:false) SnapCreateDur 10.925µs SnapOpenDur 485.086272ms 2016-04-07T04:23:32.585-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30480000 QueuedCount 1344 2016-04-07T04:23:33.517-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.752µs CommitTime 797.009616ms TotalFlushTime 31m51.384725794s TotalCommitTime 3m10.823542514s 2016-04-07T04:23:33.519-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40630528, 30315264, 332 committed:true 2016-04-07T04:23:33.519-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40630528, 30315264, 332 committed:true) SnapCreateDur 810.313689ms SnapOpenDur 59.702µs 2016-04-07T04:23:33.544-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:33.57-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:33.595-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:33.62-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:33.644-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:34.343-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30490000 QueuedCount 39772 2016-04-07T04:23:35.218-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30500000 QueuedCount 29772 2016-04-07T04:23:36.097-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30510000 QueuedCount 19772 2016-04-07T04:23:36.902-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30520000 QueuedCount 9772 2016-04-07T04:23:37.848-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40725852, 30362926, 0 committed:false 2016-04-07T04:23:38.281-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40725852, 30362926, 0 committed:false) SnapCreateDur 20.403µs SnapOpenDur 433.255533ms 2016-04-07T04:23:38.31-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30530000 QueuedCount 9796 2016-04-07T04:23:39.991-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.347µs CommitTime 803.058298ms TotalFlushTime 31m56.536995275s TotalCommitTime 3m11.626600812s 2016-04-07T04:23:39.992-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40745538, 30372769, 333 committed:true 2016-04-07T04:23:39.992-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40745538, 30372769, 333 committed:true) SnapCreateDur 815.198409ms SnapOpenDur 44.209µs 2016-04-07T04:23:40.017-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:40.043-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:40.067-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:40.075-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30540000 QueuedCount 27766 2016-04-07T04:23:40.092-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:40.117-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:44.013-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30550000 QueuedCount 40828 2016-04-07T04:23:44.969-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30560000 QueuedCount 30828 2016-04-07T04:23:45.841-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30570000 QueuedCount 20828 2016-04-07T04:23:46.707-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30580000 QueuedCount 10828 2016-04-07T04:23:47.657-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30590000 QueuedCount 828 2016-04-07T04:23:47.778-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40846006, 30423003, 0 committed:false 2016-04-07T04:23:48.181-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40846006, 30423003, 0 committed:false) SnapCreateDur 12.623µs SnapOpenDur 402.933747ms 2016-04-07T04:23:52.646-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 14.262µs CommitTime 798.324278ms TotalFlushTime 32m7.898164394s TotalCommitTime 3m12.42492509s 2016-04-07T04:23:52.646-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 40856318, 30428159, 334 committed:true 2016-04-07T04:23:52.646-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 40856318, 30428159, 334 committed:true) SnapCreateDur 811.376219ms SnapOpenDur 47.315µs 2016-04-07T04:23:52.67-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:52.695-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:52.722-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:52.751-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:52.996-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30600000 QueuedCount 50000 2016-04-07T04:23:54.099-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30610000 QueuedCount 50000 2016-04-07T04:23:54.409-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:23:55.102-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30620000 QueuedCount 50000 2016-04-07T04:23:56.06-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30630000 QueuedCount 43371 2016-04-07T04:23:56.912-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30640000 QueuedCount 33371 2016-04-07T04:23:57.706-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30650000 QueuedCount 23371 2016-04-07T04:23:58.686-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30660000 QueuedCount 13371 2016-04-07T04:23:59.66-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30670000 QueuedCount 3371 2016-04-07T04:24:00.747-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.998µs CommitTime 824.51028ms TotalFlushTime 32m15.043781532s TotalCommitTime 3m13.24943537s 2016-04-07T04:24:00.747-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41008510, 30504255, 335 committed:true 2016-04-07T04:24:00.747-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41008510, 30504255, 335 committed:true) SnapCreateDur 835.498529ms SnapOpenDur 52.838µs 2016-04-07T04:24:01.359-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30680000 QueuedCount 9196 2016-04-07T04:24:02.642-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.355µs CommitTime 396.418232ms TotalFlushTime 32m16.500529703s TotalCommitTime 3m13.645853602s 2016-04-07T04:24:02.642-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41039668, 30519834, 336 committed:true 2016-04-07T04:24:02.642-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41039668, 30519834, 336 committed:true) SnapCreateDur 407.705399ms SnapOpenDur 49.773µs 2016-04-07T04:24:02.667-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:02.68-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30690000 QueuedCount 11176 2016-04-07T04:24:02.692-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:02.716-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:02.741-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:02.765-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:03.606-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30700000 QueuedCount 45789 2016-04-07T04:24:04.644-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30710000 QueuedCount 35789 2016-04-07T04:24:05.556-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30720000 QueuedCount 25789 2016-04-07T04:24:06.554-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30730000 QueuedCount 15789 2016-04-07T04:24:07.403-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30740000 QueuedCount 5789 2016-04-07T04:24:07.862-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41151048, 30575524, 0 committed:false 2016-04-07T04:24:08.336-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41151048, 30575524, 0 committed:false) SnapCreateDur 18.263µs SnapOpenDur 474.183209ms 2016-04-07T04:24:08.741-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30750000 QueuedCount 4597 2016-04-07T04:24:09.899-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.432µs CommitTime 797.717854ms TotalFlushTime 32m22.38898743s TotalCommitTime 3m14.443571456s 2016-04-07T04:24:09.899-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41168406, 30584203, 337 committed:true 2016-04-07T04:24:09.899-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41168406, 30584203, 337 committed:true) SnapCreateDur 808.481768ms SnapOpenDur 64.163µs 2016-04-07T04:24:09.924-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:09.948-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:09.973-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:09.998-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:10.022-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:10.362-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30760000 QueuedCount 49905 2016-04-07T04:24:11.484-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30770000 QueuedCount 39905 2016-04-07T04:24:12.443-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30780000 QueuedCount 29905 2016-04-07T04:24:12.953-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":5965253,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3262412062,"bucket-1:myindex:avg_ts_items_count":26687,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8527849506,"bucket-1:myindex:delete_bytes":455120729,"bucket-1:myindex:disk_size":98277724160,"bucket-1:myindex:flush_queue_size":24053,"bucket-1:myindex:frag_percent":91,"bucket-1:myindex:get_bytes":462447671,"bucket-1:myindex:insert_bytes":3211341315,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":337,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":31345085,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":8654915,"bucket-1:myindex:num_flush_queued":30809905,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":514,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1562986028,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"337 194443571456 28485692241690922","bucket-1:myindex:timings/storage_del":"10614939 337168536199 8853502361297100019","bucket-1:myindex:timings/storage_get":"30785853 243816879411 1079854246710799393","bucket-1:myindex:timings/storage_handle_open":"337 8088308 207888668190","bucket-1:myindex:timings/storage_info":"1703 15213591 387588189589","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"340 214011130 3692697020666030","bucket-1:myindex:timings/storage_meta_set":"337 473800568 107880411438356314","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"61229878 1344284432814 5595834473619090780","bucket-1:myindex:timings/storage_snapshot_close":"177 7507180202 333635254145472322","bucket-1:myindex:timings/storage_snapshot_create":"177 68051048364 -1317116228838686552","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":428,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"6442 1023053109 185837604257497"} 2016-04-07T04:24:13.31-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30790000 QueuedCount 19905 2016-04-07T04:24:14.149-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30800000 QueuedCount 9905 2016-04-07T04:24:15.011-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41277266, 30638633, 0 committed:false 2016-04-07T04:24:15.418-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41277266, 30638633, 0 committed:false) SnapCreateDur 10.428µs SnapOpenDur 407.717857ms 2016-04-07T04:24:15.512-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30810000 QueuedCount 8726 2016-04-07T04:24:25.562-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.74µs CommitTime 887.438629ms TotalFlushTime 32m36.661769847s TotalCommitTime 3m15.331010085s 2016-04-07T04:24:25.562-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41294700, 30647350, 338 committed:true 2016-04-07T04:24:25.562-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41294700, 30647350, 338 committed:true) SnapCreateDur 898.78945ms SnapOpenDur 41.743µs 2016-04-07T04:24:25.608-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30820000 QueuedCount 7178 2016-04-07T04:24:26.618-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.783µs CommitTime 220.349343ms TotalFlushTime 32m37.46160664s TotalCommitTime 3m15.551359428s 2016-04-07T04:24:26.618-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41311332, 30655666, 339 committed:true 2016-04-07T04:24:26.618-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41311332, 30655666, 339 committed:true) SnapCreateDur 231.441647ms SnapOpenDur 51.866µs 2016-04-07T04:24:26.642-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:26.665-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:26.689-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:26.713-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:26.736-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:26.928-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30830000 QueuedCount 46442 2016-04-07T04:24:27.828-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30840000 QueuedCount 36442 2016-04-07T04:24:28.766-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30850000 QueuedCount 26442 2016-04-07T04:24:31.749-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30860000 QueuedCount 16442 2016-04-07T04:24:32.592-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30870000 QueuedCount 6442 2016-04-07T04:24:33.284-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41408250, 30704125, 0 committed:false 2016-04-07T04:24:33.751-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41408250, 30704125, 0 committed:false) SnapCreateDur 9.36µs SnapOpenDur 467.541701ms 2016-04-07T04:24:34.029-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30880000 QueuedCount 6774 2016-04-07T04:24:35.465-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.545µs CommitTime 816.416165ms TotalFlushTime 32m44.926654278s TotalCommitTime 3m16.367775593s 2016-04-07T04:24:35.465-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41428548, 30714274, 340 committed:true 2016-04-07T04:24:35.465-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41428548, 30714274, 340 committed:true) SnapCreateDur 827.584258ms SnapOpenDur 54.364µs 2016-04-07T04:24:35.492-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:35.516-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:35.54-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:35.563-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:35.585-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:35.774-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30890000 QueuedCount 37153 2016-04-07T04:24:36.684-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30900000 QueuedCount 27153 2016-04-07T04:24:37.623-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30910000 QueuedCount 17153 2016-04-07T04:24:38.426-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30920000 QueuedCount 7153 2016-04-07T04:24:39.05-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41507968, 30753984, 0 committed:false 2016-04-07T04:24:39.493-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41507968, 30753984, 0 committed:false) SnapCreateDur 11.217µs SnapOpenDur 443.060611ms 2016-04-07T04:24:39.923-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30930000 QueuedCount 7328 2016-04-07T04:24:41.262-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.969µs CommitTime 742.795282ms TotalFlushTime 32m49.455500551s TotalCommitTime 3m17.110570875s 2016-04-07T04:24:41.262-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41527932, 30763966, 341 committed:true 2016-04-07T04:24:41.262-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41527932, 30763966, 341 committed:true) SnapCreateDur 753.266802ms SnapOpenDur 57.299µs 2016-04-07T04:24:41.289-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:41.314-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:41.338-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:41.364-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:41.389-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:41.467-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30940000 QueuedCount 50000 2016-04-07T04:24:42.518-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30950000 QueuedCount 46625 2016-04-07T04:24:43.37-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30960000 QueuedCount 36625 2016-04-07T04:24:44.266-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30970000 QueuedCount 26625 2016-04-07T04:24:45.079-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30980000 QueuedCount 16625 2016-04-07T04:24:45.904-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 30990000 QueuedCount 6625 2016-04-07T04:24:46.595-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41644502, 30822251, 0 committed:false 2016-04-07T04:24:47.014-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41644502, 30822251, 0 committed:false) SnapCreateDur 16.32µs SnapOpenDur 418.677981ms 2016-04-07T04:24:47.453-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31000000 QueuedCount 7756 2016-04-07T04:24:49.146-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 5.955µs CommitTime 955.477902ms TotalFlushTime 32m55.853167683s TotalCommitTime 3m18.066048777s 2016-04-07T04:24:49.148-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41666422, 30833211, 342 committed:true 2016-04-07T04:24:49.148-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41666422, 30833211, 342 committed:true) SnapCreateDur 967.340069ms SnapOpenDur 48.283µs 2016-04-07T04:24:49.173-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:49.198-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:49.224-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:49.248-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:49.273-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:24:49.787-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31010000 QueuedCount 50000 2016-04-07T04:24:51.033-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31020000 QueuedCount 47783 2016-04-07T04:24:52.071-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31030000 QueuedCount 37783 2016-04-07T04:24:53.078-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31040000 QueuedCount 27783 2016-04-07T04:24:53.933-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31050000 QueuedCount 17783 2016-04-07T04:24:54.78-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31060000 QueuedCount 7783 2016-04-07T04:24:55.366-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41784618, 30892309, 0 committed:false 2016-04-07T04:24:55.795-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41784618, 30892309, 0 committed:false) SnapCreateDur 11.724µs SnapOpenDur 429.417551ms 2016-04-07T04:24:56.096-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31070000 QueuedCount 6424 2016-04-07T04:25:00.337-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.076µs CommitTime 845.851528ms TotalFlushTime 33m5.654285246s TotalCommitTime 3m18.911900305s 2016-04-07T04:25:00.337-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41801644, 30900822, 343 committed:true 2016-04-07T04:25:00.337-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41801644, 30900822, 343 committed:true) SnapCreateDur 857.355649ms SnapOpenDur 41.914µs 2016-04-07T04:25:00.362-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:00.387-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:00.412-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:00.436-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:00.461-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:00.658-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31080000 QueuedCount 50000 2016-04-07T04:25:01.616-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31090000 QueuedCount 42548 2016-04-07T04:25:02.682-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31100000 QueuedCount 32548 2016-04-07T04:25:03.565-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31110000 QueuedCount 22548 2016-04-07T04:25:08.526-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31120000 QueuedCount 12548 2016-04-07T04:25:09.424-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31130000 QueuedCount 2548 2016-04-07T04:25:09.685-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41912096, 30956048, 0 committed:false 2016-04-07T04:25:10.223-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41912096, 30956048, 0 committed:false) SnapCreateDur 61.349µs SnapOpenDur 538.711231ms 2016-04-07T04:25:10.819-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31140000 QueuedCount 1753 2016-04-07T04:25:11.856-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 8.609µs CommitTime 906.415975ms TotalFlushTime 33m15.627340767s TotalCommitTime 3m19.81831628s 2016-04-07T04:25:11.856-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 41930234, 30965117, 344 committed:true 2016-04-07T04:25:11.856-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 41930234, 30965117, 344 committed:true) SnapCreateDur 916.778699ms SnapOpenDur 47.887µs 2016-04-07T04:25:11.881-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:11.905-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:11.927-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:11.951-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:12.699-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31150000 QueuedCount 50000 2016-04-07T04:25:12.839-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:12.954-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":5573609,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":4471521797,"bucket-1:myindex:avg_ts_items_count":25061,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8597301701,"bucket-1:myindex:delete_bytes":471110064,"bucket-1:myindex:disk_size":101005549568,"bucket-1:myindex:flush_queue_size":51012,"bucket-1:myindex:frag_percent":91,"bucket-1:myindex:get_bytes":478699564,"bucket-1:myindex:insert_bytes":3250385040,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":344,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":31726342,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":8273658,"bucket-1:myindex:num_flush_queued":31202560,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":527,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1632438572,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"344 199818316280 4532859779243845974","bucket-1:myindex:timings/storage_del":"10974760 354418229014 2136421884811896484","bucket-1:myindex:timings/storage_get":"31151549 248015802514 1094516588708755220","bucket-1:myindex:timings/storage_handle_open":"344 8271055 213011662397","bucket-1:myindex:timings/storage_info":"1743 15576080 396110930148","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"347 216803724 3693821624415400","bucket-1:myindex:timings/storage_meta_set":"344 477241401 107882119079967139","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"61949520 1373793649664 8164301496855953658","bucket-1:myindex:timings/storage_snapshot_close":"183 7807560494 348828229546289562","bucket-1:myindex:timings/storage_snapshot_create":"183 70755680421 -86527818636553897","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":385,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"6586 1049533822 190812705094608"} 2016-04-07T04:25:13.832-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31160000 QueuedCount 50000 2016-04-07T04:25:14.779-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31170000 QueuedCount 50000 2016-04-07T04:25:15.708-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31180000 QueuedCount 50000 2016-04-07T04:25:16.623-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31190000 QueuedCount 46186 2016-04-07T04:25:17.489-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31200000 QueuedCount 36186 2016-04-07T04:25:18.421-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31210000 QueuedCount 26186 2016-04-07T04:25:19.708-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31220000 QueuedCount 16186 2016-04-07T04:25:20.619-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31230000 QueuedCount 6186 2016-04-07T04:25:21.155-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 42116052, 31058026, 0 committed:false 2016-04-07T04:25:21.79-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 42116052, 31058026, 0 committed:false) SnapCreateDur 10.355µs SnapOpenDur 635.295837ms 2016-04-07T04:25:22.167-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31240000 QueuedCount 6189 2016-04-07T04:25:23.726-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.932µs CommitTime 1.069559631s TotalFlushTime 33m25.60409638s TotalCommitTime 3m20.887875911s 2016-04-07T04:25:23.726-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 42135728, 31067864, 345 committed:true 2016-04-07T04:25:23.726-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 42135728, 31067864, 345 committed:true) SnapCreateDur 1.080627385s SnapOpenDur 59.1µs 2016-04-07T04:25:23.751-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:23.787-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:24.074-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31250000 QueuedCount 50000 2016-04-07T04:25:25.208-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31260000 QueuedCount 50000 2016-04-07T04:25:26.303-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31270000 QueuedCount 50000 2016-04-07T04:25:26.443-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:26.712-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:27.477-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31280000 QueuedCount 50000 2016-04-07T04:25:27.983-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:28.343-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31290000 QueuedCount 50000 2016-04-07T04:25:29.243-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31300000 QueuedCount 50000 2016-04-07T04:25:30.104-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31310000 QueuedCount 50000 2016-04-07T04:25:30.944-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31320000 QueuedCount 50000 2016-04-07T04:25:31.954-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31330000 QueuedCount 42721 2016-04-07T04:25:32.821-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31340000 QueuedCount 32721 2016-04-07T04:25:38.474-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31350000 QueuedCount 22721 2016-04-07T04:25:39.271-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31360000 QueuedCount 12721 2016-04-07T04:25:40.043-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31370000 QueuedCount 2721 2016-04-07T04:25:41.33-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.538µs CommitTime 1.056120195s TotalFlushTime 33m41.890881663s TotalCommitTime 3m21.943996106s 2016-04-07T04:25:41.331-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 42384630, 31192315, 346 committed:true 2016-04-07T04:25:41.331-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 42384630, 31192315, 346 committed:true) SnapCreateDur 1.066683271s SnapOpenDur 59.779µs 2016-04-07T04:25:42.502-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31380000 QueuedCount 8422 2016-04-07T04:25:47.471-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.647µs CommitTime 385.396835ms TotalFlushTime 33m47.589321856s TotalCommitTime 3m22.329392941s 2016-04-07T04:25:47.471-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 42415510, 31207755, 347 committed:true 2016-04-07T04:25:47.471-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 42415510, 31207755, 347 committed:true) SnapCreateDur 402.732727ms SnapOpenDur 46.82µs 2016-04-07T04:25:47.695-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31390000 QueuedCount 10980 2016-04-07T04:25:48.872-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31400000 QueuedCount 980 2016-04-07T04:25:49.366-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.989µs CommitTime 327.640202ms TotalFlushTime 33m49.115596091s TotalCommitTime 3m22.657033143s 2016-04-07T04:25:49.366-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 42440226, 31220113, 348 committed:true 2016-04-07T04:25:49.366-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 42440226, 31220113, 348 committed:true) SnapCreateDur 339.201166ms SnapOpenDur 58.101µs 2016-04-07T04:25:49.403-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:49.54-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:51.429-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31410000 QueuedCount 50000 2016-04-07T04:25:51.573-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:52.42-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:52.433-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31420000 QueuedCount 49415 2016-04-07T04:25:53.437-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31430000 QueuedCount 50000 2016-04-07T04:25:54.385-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31440000 QueuedCount 50000 2016-04-07T04:25:55.932-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31450000 QueuedCount 50000 2016-04-07T04:25:59.065-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31460000 QueuedCount 50000 2016-04-07T04:25:59.479-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:25:59.996-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31470000 QueuedCount 50000 2016-04-07T04:26:00.897-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31480000 QueuedCount 50000 2016-04-07T04:26:01.848-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31490000 QueuedCount 50000 2016-04-07T04:26:02.838-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31500000 QueuedCount 45516 2016-04-07T04:26:03.694-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31510000 QueuedCount 35516 2016-04-07T04:26:04.564-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31520000 QueuedCount 25516 2016-04-07T04:26:05.426-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31530000 QueuedCount 15516 2016-04-07T04:26:06.284-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31540000 QueuedCount 5516 2016-04-07T04:26:08.023-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.516µs CommitTime 1.202154124s TotalFlushTime 34m6.283335207s TotalCommitTime 3m23.859187267s 2016-04-07T04:26:08.025-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 42724644, 31362322, 349 committed:true 2016-04-07T04:26:08.025-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 42724644, 31362322, 349 committed:true) SnapCreateDur 1.213914401s SnapOpenDur 45.956µs 2016-04-07T04:26:08.29-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.16µs CommitTime 72.218948ms TotalFlushTime 34m6.443030572s TotalCommitTime 3m23.931406215s 2016-04-07T04:26:08.29-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 42728062, 31364031, 350 committed:true 2016-04-07T04:26:08.29-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 42728062, 31364031, 350 committed:true) SnapCreateDur 83.592125ms SnapOpenDur 34.119µs 2016-04-07T04:26:08.313-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:08.341-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:08.495-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31550000 QueuedCount 50000 2016-04-07T04:26:09.668-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31560000 QueuedCount 50000 2016-04-07T04:26:09.689-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:10.696-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31570000 QueuedCount 50000 2016-04-07T04:26:11.144-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:12.954-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":5145844,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":6110988527,"bucket-1:myindex:avg_ts_items_count":44598,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8677411294,"bucket-1:myindex:delete_bytes":489537198,"bucket-1:myindex:disk_size":104046256128,"bucket-1:myindex:flush_queue_size":50808,"bucket-1:myindex:frag_percent":91,"bucket-1:myindex:get_bytes":497429891,"bucket-1:myindex:insert_bytes":3295381487,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":350,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":32221237,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":7778763,"bucket-1:myindex:num_flush_queued":31630121,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":534,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":265565606,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"350 203931406215 8498476386490592269","bucket-1:myindex:timings/storage_del":"11395594 375129914742 -473059432418696846","bucket-1:myindex:timings/storage_get":"31579314 253841998977 1138364220327517049","bucket-1:myindex:timings/storage_handle_open":"350 8406119 216114691643","bucket-1:myindex:timings/storage_info":"1769 15847483 403755822129","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"353 219258346 3694846363006608","bucket-1:myindex:timings/storage_meta_set":"350 480160932 107883561963168158","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"62791188 1399996611541 -7161555615478132987","bucket-1:myindex:timings/storage_snapshot_close":"184 7887088312 355152903382130686","bucket-1:myindex:timings/storage_snapshot_create":"184 71390888067 316960934900307419","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":364,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"6730 1076821355 196145882058229"} 2016-04-07T04:26:14.697-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31580000 QueuedCount 50000 2016-04-07T04:26:15.215-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:15.609-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31590000 QueuedCount 46479 2016-04-07T04:26:16.508-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31600000 QueuedCount 36479 2016-04-07T04:26:21.857-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31610000 QueuedCount 26479 2016-04-07T04:26:22.759-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31620000 QueuedCount 16479 2016-04-07T04:26:23.615-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31630000 QueuedCount 6479 2016-04-07T04:26:25.213-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.478µs CommitTime 922.623405ms TotalFlushTime 34m22.295161561s TotalCommitTime 3m24.85402962s 2016-04-07T04:26:25.213-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 42903602, 31451801, 351 committed:true 2016-04-07T04:26:25.213-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 42903602, 31451801, 351 committed:true) SnapCreateDur 933.093926ms SnapOpenDur 56.939µs 2016-04-07T04:26:25.553-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31640000 QueuedCount 43339 2016-04-07T04:26:26.547-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31650000 QueuedCount 33339 2016-04-07T04:26:27.459-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31660000 QueuedCount 23339 2016-04-07T04:26:28.515-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31670000 QueuedCount 13339 2016-04-07T04:26:29.439-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31680000 QueuedCount 3339 2016-04-07T04:26:30.539-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.043µs CommitTime 770.746395ms TotalFlushTime 34m26.784908145s TotalCommitTime 3m25.624776015s 2016-04-07T04:26:30.539-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 42995918, 31497959, 352 committed:true 2016-04-07T04:26:30.539-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 42995918, 31497959, 352 committed:true) SnapCreateDur 782.57575ms SnapOpenDur 30.5µs 2016-04-07T04:26:31.146-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.448µs CommitTime 113.301457ms TotalFlushTime 34m27.242996594s TotalCommitTime 3m25.738077472s 2016-04-07T04:26:31.146-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 43004118, 31502059, 353 committed:true 2016-04-07T04:26:31.146-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 43004118, 31502059, 353 committed:true) SnapCreateDur 124.581251ms SnapOpenDur 33.851µs 2016-04-07T04:26:31.168-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:31.194-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:31.345-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31690000 QueuedCount 50000 2016-04-07T04:26:31.793-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:32.543-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31700000 QueuedCount 50000 2016-04-07T04:26:33.519-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31710000 QueuedCount 50000 2016-04-07T04:26:34.545-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31720000 QueuedCount 50000 2016-04-07T04:26:34.806-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:35.454-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31730000 QueuedCount 50000 2016-04-07T04:26:36.084-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:36.537-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31740000 QueuedCount 50000 2016-04-07T04:26:37.65-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31750000 QueuedCount 40042 2016-04-07T04:26:38.78-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31760000 QueuedCount 30042 2016-04-07T04:26:40.003-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31770000 QueuedCount 20042 2016-04-07T04:26:41.143-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31780000 QueuedCount 10042 2016-04-07T04:26:42.315-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31790000 QueuedCount 42 2016-04-07T04:26:43.384-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.653µs CommitTime 1.054656606s TotalFlushTime 34m38.239400053s TotalCommitTime 3m26.792734078s 2016-04-07T04:26:43.384-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 43205914, 31602957, 354 committed:true 2016-04-07T04:26:43.384-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 43205914, 31602957, 354 committed:true) SnapCreateDur 1.065813368s SnapOpenDur 52.665µs 2016-04-07T04:26:45.312-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31800000 QueuedCount 33500 2016-04-07T04:26:46.43-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31810000 QueuedCount 23500 2016-04-07T04:26:51.387-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31820000 QueuedCount 13500 2016-04-07T04:26:52.402-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31830000 QueuedCount 3500 2016-04-07T04:26:53.526-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.195µs CommitTime 688.389238ms TotalFlushTime 34m47.635246324s TotalCommitTime 3m27.481123316s 2016-04-07T04:26:53.526-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 43291488, 31645744, 355 committed:true 2016-04-07T04:26:53.526-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 43291488, 31645744, 355 committed:true) SnapCreateDur 699.553008ms SnapOpenDur 61.1µs 2016-04-07T04:26:54.43-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.827µs CommitTime 219.373916ms TotalFlushTime 34m48.283142334s TotalCommitTime 3m27.700497232s 2016-04-07T04:26:54.43-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 43304106, 31652053, 356 committed:true 2016-04-07T04:26:54.43-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 43304106, 31652053, 356 committed:true) SnapCreateDur 230.38888ms SnapOpenDur 46.077µs 2016-04-07T04:26:54.456-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31840000 QueuedCount 10650 2016-04-07T04:26:54.457-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:54.479-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:26:54.506-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:00.041-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31850000 QueuedCount 50000 2016-04-07T04:27:00.971-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:01.128-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31860000 QueuedCount 50000 2016-04-07T04:27:02.267-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31870000 QueuedCount 50000 2016-04-07T04:27:06.627-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31880000 QueuedCount 50000 2016-04-07T04:27:07.935-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31890000 QueuedCount 45574 2016-04-07T04:27:08.994-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31900000 QueuedCount 35574 2016-04-07T04:27:10.078-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31910000 QueuedCount 25574 2016-04-07T04:27:11.073-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31920000 QueuedCount 15574 2016-04-07T04:27:12.011-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31930000 QueuedCount 5574 2016-04-07T04:27:12.954-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":4839584,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":5081649263,"bucket-1:myindex:avg_ts_items_count":30698,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8741902127,"bucket-1:myindex:delete_bytes":504271062,"bucket-1:myindex:disk_size":106358517760,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":91,"bucket-1:myindex:get_bytes":512396987,"bucket-1:myindex:insert_bytes":3331359570,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":356,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":32520974,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":7479026,"bucket-1:myindex:num_flush_queued":31935574,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":540,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":904283857,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"356 207700497232 -6855841299606511512","bucket-1:myindex:timings/storage_del":"11746326 390696486687 -8059208723442127425","bucket-1:myindex:timings/storage_get":"31935574 261209118328 1232418627024263034","bucket-1:myindex:timings/storage_handle_open":"356 8550325 219769871439","bucket-1:myindex:timings/storage_info":"1796 16078533 408717328805","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"360 222429003 3696327632077155","bucket-1:myindex:timings/storage_meta_set":"357 483697441 107885362729814091","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"63492652 1433426083094 5395907702562498218","bucket-1:myindex:timings/storage_snapshot_close":"184 7887088312 355152903382130686","bucket-1:myindex:timings/storage_snapshot_create":"184 71390888067 316960934900307419","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":348,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"6874 1102906035 200989389387927"} 2016-04-07T04:27:13.813-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.224µs CommitTime 1.216780239s TotalFlushTime 35m6.277097177s TotalCommitTime 3m28.917277471s 2016-04-07T04:27:13.813-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 43492652, 31746326, 357 committed:true 2016-04-07T04:27:13.813-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 43492652, 31746326, 357 committed:true) SnapCreateDur 1.227239009s SnapOpenDur 47.095µs 2016-04-07T04:27:14.338-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31940000 QueuedCount 18448 2016-04-07T04:27:15.442-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31950000 QueuedCount 8448 2016-04-07T04:27:16.84-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.56µs CommitTime 454.396031ms TotalFlushTime 35m8.796618789s TotalCommitTime 3m29.371673502s 2016-04-07T04:27:16.84-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 43537718, 31768859, 358 committed:true 2016-04-07T04:27:16.84-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 43537718, 31768859, 358 committed:true) SnapCreateDur 465.69821ms SnapOpenDur 38.09µs 2016-04-07T04:27:16.878-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:17.028-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31960000 QueuedCount 50000 2016-04-07T04:27:18.158-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31970000 QueuedCount 50000 2016-04-07T04:27:18.313-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:19.551-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31980000 QueuedCount 50000 2016-04-07T04:27:19.84-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:20.628-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 31990000 QueuedCount 50000 2016-04-07T04:27:21.758-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32000000 QueuedCount 43795 2016-04-07T04:27:22.877-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32010000 QueuedCount 33795 2016-04-07T04:27:23.894-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32020000 QueuedCount 23795 2016-04-07T04:27:32.181-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32030000 QueuedCount 13795 2016-04-07T04:27:33.216-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32040000 QueuedCount 3795 2016-04-07T04:27:34.657-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.675µs CommitTime 1.030850808s TotalFlushTime 35m25.44975532s TotalCommitTime 3m30.40252431s 2016-04-07T04:27:34.657-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 43705830, 31852915, 359 committed:true 2016-04-07T04:27:34.657-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 43705830, 31852915, 359 committed:true) SnapCreateDur 1.041542768s SnapOpenDur 53.367µs 2016-04-07T04:27:35.358-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32050000 QueuedCount 5081 2016-04-07T04:27:36.274-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 13.716µs CommitTime 291.48447ms TotalFlushTime 35m26.729543095s TotalCommitTime 3m30.69400878s 2016-04-07T04:27:36.275-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 43728050, 31864025, 360 committed:true 2016-04-07T04:27:36.275-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 43728050, 31864025, 360 committed:true) SnapCreateDur 301.964148ms SnapOpenDur 51.691µs 2016-04-07T04:27:36.313-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:36.828-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32060000 QueuedCount 50000 2016-04-07T04:27:37.974-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32070000 QueuedCount 50000 2016-04-07T04:27:38.753-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:39.503-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32080000 QueuedCount 50000 2016-04-07T04:27:41.473-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32090000 QueuedCount 50000 2016-04-07T04:27:41.486-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:43.716-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32100000 QueuedCount 46868 2016-04-07T04:27:44.718-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32110000 QueuedCount 36868 2016-04-07T04:27:45.742-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32120000 QueuedCount 26868 2016-04-07T04:27:46.748-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32130000 QueuedCount 16868 2016-04-07T04:27:47.833-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32140000 QueuedCount 6868 2016-04-07T04:27:49.483-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.31µs CommitTime 1.015115291s TotalFlushTime 35m38.766558435s TotalCommitTime 3m31.709124071s 2016-04-07T04:27:49.483-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 43908704, 31954352, 361 committed:true 2016-04-07T04:27:49.483-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 43908704, 31954352, 361 committed:true) SnapCreateDur 1.025802597s SnapOpenDur 35.932µs 2016-04-07T04:27:50.046-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32150000 QueuedCount 17928 2016-04-07T04:27:51.79-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32160000 QueuedCount 7928 2016-04-07T04:27:53.198-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.823µs CommitTime 415.180542ms TotalFlushTime 35m42.014533903s TotalCommitTime 3m32.124304613s 2016-04-07T04:27:53.199-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 43950110, 31975055, 362 committed:true 2016-04-07T04:27:53.199-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 43950110, 31975055, 362 committed:true) SnapCreateDur 426.614026ms SnapOpenDur 44.375µs 2016-04-07T04:27:53.224-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:53.26-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:53.402-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32170000 QueuedCount 50000 2016-04-07T04:27:54.23-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:54.421-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:54.578-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32180000 QueuedCount 50000 2016-04-07T04:27:55.707-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32190000 QueuedCount 50000 2016-04-07T04:27:56.901-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32200000 QueuedCount 50000 2016-04-07T04:27:57.385-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:27:58.033-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32210000 QueuedCount 50000 2016-04-07T04:27:59.056-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32220000 QueuedCount 40061 2016-04-07T04:28:00.117-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32230000 QueuedCount 30061 2016-04-07T04:28:01.117-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32240000 QueuedCount 20061 2016-04-07T04:28:08.545-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32250000 QueuedCount 10061 2016-04-07T04:28:09.535-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32260000 QueuedCount 61 2016-04-07T04:28:10.57-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.26µs CommitTime 1.013791443s TotalFlushTime 35m58.206442717s TotalCommitTime 3m33.138096056s 2016-04-07T04:28:10.57-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44131480, 32065740, 363 committed:true 2016-04-07T04:28:10.57-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44131480, 32065740, 363 committed:true) SnapCreateDur 1.025025883s SnapOpenDur 35.425µs 2016-04-07T04:28:11.725-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32270000 QueuedCount 1371 2016-04-07T04:28:12.158-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.952µs CommitTime 261.655965ms TotalFlushTime 35m59.489900357s TotalCommitTime 3m33.399752021s 2016-04-07T04:28:12.158-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44153702, 32076851, 364 committed:true 2016-04-07T04:28:12.158-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44153702, 32076851, 364 committed:true) SnapCreateDur 272.944547ms SnapOpenDur 46.12µs 2016-04-07T04:28:12.19-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:12.212-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:12.233-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:12.257-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:12.278-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:12.954-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":4446827,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":6875133013,"bucket-1:myindex:avg_ts_items_count":39416,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8806042561,"bucket-1:myindex:delete_bytes":519002389,"bucket-1:myindex:disk_size":109567795200,"bucket-1:myindex:flush_queue_size":50620,"bucket-1:myindex:frag_percent":91,"bucket-1:myindex:get_bytes":527361159,"bucket-1:myindex:insert_bytes":3367331310,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":364,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":32874421,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":7125579,"bucket-1:myindex:num_flush_queued":32328950,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":548,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1588233901,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"364 213399752021 -1722123941594095747","bucket-1:myindex:timings/storage_del":"12083705 405058104123 3027415783677964713","bucket-1:myindex:timings/storage_get":"32278331 270946548221 1494985019354990943","bucket-1:myindex:timings/storage_handle_open":"364 8719436 223402630848","bucket-1:myindex:timings/storage_info":"1825 16445873 423195847785","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"367 225244559 3697488885292949","bucket-1:myindex:timings/storage_meta_set":"364 486741308 107886707906500682","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"64167408 1462999663762 -9158520288861154244","bucket-1:myindex:timings/storage_snapshot_close":"184 7887088312 355152903382130686","bucket-1:myindex:timings/storage_snapshot_create":"184 71390888067 316960934900307419","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":324,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"7016 1129029464 206145058996672"} 2016-04-07T04:28:13.235-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32280000 QueuedCount 50000 2016-04-07T04:28:14.377-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32290000 QueuedCount 50000 2016-04-07T04:28:15.463-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32300000 QueuedCount 40372 2016-04-07T04:28:17.608-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32310000 QueuedCount 30372 2016-04-07T04:28:18.775-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32320000 QueuedCount 20372 2016-04-07T04:28:19.772-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32330000 QueuedCount 10372 2016-04-07T04:28:20.834-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32340000 QueuedCount 372 2016-04-07T04:28:20.846-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44289638, 32144819, 0 committed:false 2016-04-07T04:28:21.352-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44289638, 32144819, 0 committed:false) SnapCreateDur 23.553µs SnapOpenDur 506.820176ms 2016-04-07T04:28:22.343-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32350000 QueuedCount 6802 2016-04-07T04:28:24.043-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.178µs CommitTime 942.59347ms TotalFlushTime 36m9.791311596s TotalCommitTime 3m34.342345491s 2016-04-07T04:28:24.043-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44321968, 32160984, 365 committed:true 2016-04-07T04:28:24.043-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44321968, 32160984, 365 committed:true) SnapCreateDur 952.963645ms SnapOpenDur 42.033µs 2016-04-07T04:28:24.075-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:24.101-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:24.125-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:24.147-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:24.42-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32360000 QueuedCount 50000 2016-04-07T04:28:28.174-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:28.3-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32370000 QueuedCount 50000 2016-04-07T04:28:29.496-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32380000 QueuedCount 50000 2016-04-07T04:28:30.621-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32390000 QueuedCount 50000 2016-04-07T04:28:31.754-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32400000 QueuedCount 45182 2016-04-07T04:28:32.761-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32410000 QueuedCount 35182 2016-04-07T04:28:33.721-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32420000 QueuedCount 25182 2016-04-07T04:28:34.765-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32430000 QueuedCount 15182 2016-04-07T04:28:35.735-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32440000 QueuedCount 5182 2016-04-07T04:28:37.213-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 10.584µs CommitTime 915.626635ms TotalFlushTime 36m21.887185894s TotalCommitTime 3m35.257972126s 2016-04-07T04:28:37.213-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44495882, 32247941, 366 committed:true 2016-04-07T04:28:37.213-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44495882, 32247941, 366 committed:true) SnapCreateDur 926.56045ms SnapOpenDur 59.763µs 2016-04-07T04:28:37.722-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32450000 QueuedCount 33420 2016-04-07T04:28:46.008-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32460000 QueuedCount 23420 2016-04-07T04:28:47.069-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32470000 QueuedCount 13420 2016-04-07T04:28:48.206-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32480000 QueuedCount 3420 2016-04-07T04:28:49.195-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.102µs CommitTime 676.151084ms TotalFlushTime 36m33.136273129s TotalCommitTime 3m35.93412321s 2016-04-07T04:28:49.195-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44571164, 32285582, 367 committed:true 2016-04-07T04:28:49.195-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44571164, 32285582, 367 committed:true) SnapCreateDur 687.081663ms SnapOpenDur 41.837µs 2016-04-07T04:28:49.292-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.414µs CommitTime 54.817638ms TotalFlushTime 36m33.145168668s TotalCommitTime 3m35.988940848s 2016-04-07T04:28:49.292-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44572022, 32286011, 368 committed:true 2016-04-07T04:28:49.292-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44572022, 32286011, 368 committed:true) SnapCreateDur 66.035941ms SnapOpenDur 35.293µs 2016-04-07T04:28:49.317-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:49.341-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:49.808-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:49.953-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32490000 QueuedCount 50000 2016-04-07T04:28:49.969-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:51.083-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32500000 QueuedCount 50000 2016-04-07T04:28:51.255-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:28:52.167-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32510000 QueuedCount 43863 2016-04-07T04:28:53.23-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32520000 QueuedCount 33863 2016-04-07T04:28:54.417-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32530000 QueuedCount 23863 2016-04-07T04:28:56.277-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32540000 QueuedCount 13863 2016-04-07T04:28:57.308-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32550000 QueuedCount 3863 2016-04-07T04:28:57.75-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44709742, 32354871, 0 committed:false 2016-04-07T04:28:58.273-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44709742, 32354871, 0 committed:false) SnapCreateDur 17.611µs SnapOpenDur 522.605338ms 2016-04-07T04:28:59.119-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32560000 QueuedCount 4893 2016-04-07T04:29:00.748-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.473µs CommitTime 926.278584ms TotalFlushTime 36m43.007401106s TotalCommitTime 3m36.915219432s 2016-04-07T04:29:00.748-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44731432, 32365716, 369 committed:true 2016-04-07T04:29:00.748-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44731432, 32365716, 369 committed:true) SnapCreateDur 939.775065ms SnapOpenDur 44.175µs 2016-04-07T04:29:00.784-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:01.182-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:01.406-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32570000 QueuedCount 50000 2016-04-07T04:29:02.569-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32580000 QueuedCount 50000 2016-04-07T04:29:02.888-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:03.49-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:03.775-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32590000 QueuedCount 50000 2016-04-07T04:29:04.8-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32600000 QueuedCount 41449 2016-04-07T04:29:05.838-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32610000 QueuedCount 31449 2016-04-07T04:29:06.819-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32620000 QueuedCount 21449 2016-04-07T04:29:07.801-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32630000 QueuedCount 11449 2016-04-07T04:29:08.822-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32640000 QueuedCount 1449 2016-04-07T04:29:09.842-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.625µs CommitTime 867.662421ms TotalFlushTime 36m51.105063016s TotalCommitTime 3m37.782881853s 2016-04-07T04:29:09.842-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44882042, 32441021, 370 committed:true 2016-04-07T04:29:09.842-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44882042, 32441021, 370 committed:true) SnapCreateDur 878.424244ms SnapOpenDur 52.013µs 2016-04-07T04:29:12.955-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":4114260,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":6958336576,"bucket-1:myindex:avg_ts_items_count":52941,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8875921248,"bucket-1:myindex:delete_bytes":535080433,"bucket-1:myindex:disk_size":112541683712,"bucket-1:myindex:flush_queue_size":14799,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":543700600,"bucket-1:myindex:insert_bytes":3406591755,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":370,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":33257558,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":6742442,"bucket-1:myindex:num_flush_queued":32660898,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":556,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":9093604187,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"370 217782881853 2075745996636868775","bucket-1:myindex:timings/storage_del":"12445612 421625067533 -4825272405827269949","bucket-1:myindex:timings/storage_get":"32646100 278627402711 1621607912494647313","bucket-1:myindex:timings/storage_handle_open":"370 8882469 228272008267","bucket-1:myindex:timings/storage_info":"1853 16761147 432810301745","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"373 227893165 3698669480110559","bucket-1:myindex:timings/storage_meta_set":"370 489586239 107888069394159709","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"64891224 1492483687754 330747919952063226","bucket-1:myindex:timings/storage_snapshot_close":"186 8004937460 362118011095532070","bucket-1:myindex:timings/storage_snapshot_create":"186 72420124192 846749188435599792","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":302,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"7160 1156034164 211347961810026"} 2016-04-07T04:29:13.477-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32650000 QueuedCount 10898 2016-04-07T04:29:14.57-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32660000 QueuedCount 898 2016-04-07T04:29:15.099-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.802µs CommitTime 375.569092ms TotalFlushTime 36m55.939039136s TotalCommitTime 3m38.158450945s 2016-04-07T04:29:15.099-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44920332, 32460166, 371 committed:true 2016-04-07T04:29:15.099-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44920332, 32460166, 371 committed:true) SnapCreateDur 387.015319ms SnapOpenDur 36.16µs 2016-04-07T04:29:22.398-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.308µs CommitTime 193.172374ms TotalFlushTime 37m3.007311194s TotalCommitTime 3m38.351623319s 2016-04-07T04:29:22.398-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44934550, 32467275, 372 committed:true 2016-04-07T04:29:22.398-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44934550, 32467275, 372 committed:true) SnapCreateDur 204.299555ms SnapOpenDur 43.17µs 2016-04-07T04:29:22.581-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32670000 QueuedCount 2864 2016-04-07T04:29:23.04-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.182µs CommitTime 159.086994ms TotalFlushTime 37m3.454667968s TotalCommitTime 3m38.510710313s 2016-04-07T04:29:23.04-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 44943910, 32471955, 373 committed:true 2016-04-07T04:29:23.04-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 44943910, 32471955, 373 committed:true) SnapCreateDur 169.330813ms SnapOpenDur 61.456µs 2016-04-07T04:29:23.066-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:23.091-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:23.255-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:23.881-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32680000 QueuedCount 50000 2016-04-07T04:29:24.188-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:24.925-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32690000 QueuedCount 50000 2016-04-07T04:29:25.978-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32700000 QueuedCount 50000 2016-04-07T04:29:28.3-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32710000 QueuedCount 50000 2016-04-07T04:29:29.823-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:29.843-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32720000 QueuedCount 49033 2016-04-07T04:29:30.991-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32730000 QueuedCount 50000 2016-04-07T04:29:32-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32740000 QueuedCount 47896 2016-04-07T04:29:33.002-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32750000 QueuedCount 37896 2016-04-07T04:29:34.01-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32760000 QueuedCount 27896 2016-04-07T04:29:36.172-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32770000 QueuedCount 17896 2016-04-07T04:29:37.246-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32780000 QueuedCount 7896 2016-04-07T04:29:39.18-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.439µs CommitTime 1.230393353s TotalFlushTime 37m18.151743098s TotalCommitTime 3m39.741103666s 2016-04-07T04:29:39.18-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 45170332, 32585166, 374 committed:true 2016-04-07T04:29:39.18-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 45170332, 32585166, 374 committed:true) SnapCreateDur 1.241680274s SnapOpenDur 50.705µs 2016-04-07T04:29:39.386-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.814µs CommitTime 143.546897ms TotalFlushTime 37m18.179076696s TotalCommitTime 3m39.884650563s 2016-04-07T04:29:39.386-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 45172438, 32586219, 375 committed:true 2016-04-07T04:29:39.386-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 45172438, 32586219, 375 committed:true) SnapCreateDur 155.05129ms SnapOpenDur 54.711µs 2016-04-07T04:29:39.408-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:39.431-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32790000 QueuedCount 7488 2016-04-07T04:29:39.442-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:40.546-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32800000 QueuedCount 50000 2016-04-07T04:29:41.661-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:41.803-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32810000 QueuedCount 50000 2016-04-07T04:29:41.964-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:42.861-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32820000 QueuedCount 50000 2016-04-07T04:29:43.907-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32830000 QueuedCount 50000 2016-04-07T04:29:45.015-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32840000 QueuedCount 50000 2016-04-07T04:29:46.264-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32850000 QueuedCount 50000 2016-04-07T04:29:46.938-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:47.521-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32860000 QueuedCount 50000 2016-04-07T04:29:48.534-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32870000 QueuedCount 44330 2016-04-07T04:29:52.107-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32880000 QueuedCount 34330 2016-04-07T04:29:53.049-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32890000 QueuedCount 24330 2016-04-07T04:29:54.662-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32900000 QueuedCount 14330 2016-04-07T04:29:55.747-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32910000 QueuedCount 4330 2016-04-07T04:29:57.386-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.405µs CommitTime 1.203420046s TotalFlushTime 37m34.710351283s TotalCommitTime 3m41.088070609s 2016-04-07T04:29:57.386-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 45419114, 32709557, 376 committed:true 2016-04-07T04:29:57.386-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 45419114, 32709557, 376 committed:true) SnapCreateDur 1.213763535s SnapOpenDur 39.639µs 2016-04-07T04:29:58.03-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32920000 QueuedCount 3520 2016-04-07T04:29:58.652-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 2.178µs CommitTime 289.72997ms TotalFlushTime 37m35.645572099s TotalCommitTime 3m41.377800579s 2016-04-07T04:29:58.652-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 45437218, 32718609, 377 committed:true 2016-04-07T04:29:58.652-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 45437218, 32718609, 377 committed:true) SnapCreateDur 301.167692ms SnapOpenDur 45.37µs 2016-04-07T04:29:58.675-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:58.708-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:58.729-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:58.752-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:29:58.848-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:00.071-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32930000 QueuedCount 50000 2016-04-07T04:30:04.476-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32940000 QueuedCount 50000 2016-04-07T04:30:05.708-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32950000 QueuedCount 50000 2016-04-07T04:30:06.709-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32960000 QueuedCount 49963 2016-04-07T04:30:07.719-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32970000 QueuedCount 41583 2016-04-07T04:30:08.731-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32980000 QueuedCount 31583 2016-04-07T04:30:09.737-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 32990000 QueuedCount 21583 2016-04-07T04:30:10.81-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33000000 QueuedCount 11583 2016-04-07T04:30:11.801-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33010000 QueuedCount 1583 2016-04-07T04:30:11.942-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 45610556, 32805278, 0 committed:false 2016-04-07T04:30:12.538-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 45610556, 32805278, 0 committed:false) SnapCreateDur 18.332µs SnapOpenDur 596.382443ms 2016-04-07T04:30:12.955-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":3729845,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":10141758939,"bucket-1:myindex:avg_ts_items_count":66074,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":8942430781,"bucket-1:myindex:delete_bytes":550428294,"bucket-1:myindex:disk_size":114533203968,"bucket-1:myindex:flush_queue_size":30160,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":559295926,"bucket-1:myindex:insert_bytes":3444069090,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":377,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":33592798,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":6407202,"bucket-1:myindex:num_flush_queued":33045313,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":564,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":13886629082,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"377 221377800579 5346059154615079385","bucket-1:myindex:timings/storage_del":"12808786 434826977400 -1970494316326529942","bucket-1:myindex:timings/storage_get":"33015154 287239153919 1866850056115172479","bucket-1:myindex:timings/storage_handle_open":"377 9050663 232623039573","bucket-1:myindex:timings/storage_info":"1883 17049063 439619662943","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"380 230468029 3699625872741047","bucket-1:myindex:timings/storage_meta_set":"377 492920791 107889703004233161","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"65617572 1525308985743 6097748566914340285","bucket-1:myindex:timings/storage_snapshot_close":"186 8004937460 362118011095532070","bucket-1:myindex:timings/storage_snapshot_create":"187 73016412291 1202308685444633593","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":279,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"7304 1183083491 216559521596621"} 2016-04-07T04:30:13.402-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33020000 QueuedCount 25313 2016-04-07T04:30:14.374-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33030000 QueuedCount 15313 2016-04-07T04:30:15.366-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33040000 QueuedCount 5313 2016-04-07T04:30:17.071-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.347µs CommitTime 1.149175973s TotalFlushTime 37m52.127391391s TotalCommitTime 3m42.526976552s 2016-04-07T04:30:17.071-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 45676954, 32838477, 378 committed:true 2016-04-07T04:30:17.071-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 45676954, 32838477, 378 committed:true) SnapCreateDur 1.160408118s SnapOpenDur 49.368µs 2016-04-07T04:30:17.093-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:17.117-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:17.138-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:17.161-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:17.188-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:17.636-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33050000 QueuedCount 33977 2016-04-07T04:30:20.067-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33060000 QueuedCount 23977 2016-04-07T04:30:23.805-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33070000 QueuedCount 13977 2016-04-07T04:30:24.955-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33080000 QueuedCount 3977 2016-04-07T04:30:25.261-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 45753140, 32876570, 0 committed:false 2016-04-07T04:30:25.592-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 45753140, 32876570, 0 committed:false) SnapCreateDur 19.335µs SnapOpenDur 331.281556ms 2016-04-07T04:30:26.634-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.285µs CommitTime 698.604075ms TotalFlushTime 38m0.575878777s TotalCommitTime 3m43.225580627s 2016-04-07T04:30:26.634-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 45758870, 32879435, 379 committed:true 2016-04-07T04:30:26.634-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 45758870, 32879435, 379 committed:true) SnapCreateDur 709.420972ms SnapOpenDur 42.867µs 2016-04-07T04:30:26.668-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:26.689-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:26.714-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:26.992-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33090000 QueuedCount 50000 2016-04-07T04:30:28.085-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33100000 QueuedCount 50000 2016-04-07T04:30:32.907-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33110000 QueuedCount 50000 2016-04-07T04:30:33.757-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:34.076-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33120000 QueuedCount 50000 2016-04-07T04:30:35.257-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33130000 QueuedCount 50000 2016-04-07T04:30:36.373-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33140000 QueuedCount 50000 2016-04-07T04:30:42.221-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33150000 QueuedCount 44087 2016-04-07T04:30:43.228-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33160000 QueuedCount 34087 2016-04-07T04:30:44.231-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33170000 QueuedCount 24087 2016-04-07T04:30:45.292-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33180000 QueuedCount 14087 2016-04-07T04:30:46.314-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33190000 QueuedCount 4087 2016-04-07T04:30:47.721-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.124µs CommitTime 985.482083ms TotalFlushTime 38m20.478013584s TotalCommitTime 3m44.21106271s 2016-04-07T04:30:47.721-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 45969768, 32984884, 380 committed:true 2016-04-07T04:30:47.721-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 45969768, 32984884, 380 committed:true) SnapCreateDur 997.045113ms SnapOpenDur 32.111µs 2016-04-07T04:30:48.419-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33200000 QueuedCount 751 2016-04-07T04:30:48.699-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.168µs CommitTime 253.658118ms TotalFlushTime 38m21.166588949s TotalCommitTime 3m44.464720828s 2016-04-07T04:30:48.699-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 45982892, 32991446, 381 committed:true 2016-04-07T04:30:48.699-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 45982892, 32991446, 381 committed:true) SnapCreateDur 264.229257ms SnapOpenDur 35.632µs 2016-04-07T04:30:48.722-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:48.744-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:48.765-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:48.791-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:48.82-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:49.77-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33210000 QueuedCount 50000 2016-04-07T04:30:50.938-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33220000 QueuedCount 47477 2016-04-07T04:30:52.159-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33230000 QueuedCount 37477 2016-04-07T04:30:53.147-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33240000 QueuedCount 27477 2016-04-07T04:30:54.217-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33250000 QueuedCount 17477 2016-04-07T04:30:55.254-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33260000 QueuedCount 7477 2016-04-07T04:30:56.141-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46114240, 33057120, 0 committed:false 2016-04-07T04:30:56.629-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46114240, 33057120, 0 committed:false) SnapCreateDur 10.486µs SnapOpenDur 488.138527ms 2016-04-07T04:30:56.816-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33270000 QueuedCount 2706 2016-04-07T04:30:58.185-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.243µs CommitTime 1.061608433s TotalFlushTime 38m28.974620183s TotalCommitTime 3m45.526329261s 2016-04-07T04:30:58.185-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46124544, 33062272, 382 committed:true 2016-04-07T04:30:58.185-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46124544, 33062272, 382 committed:true) SnapCreateDur 1.072788935s SnapOpenDur 64.231µs 2016-04-07T04:30:58.219-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:58.882-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:59.06-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:30:59.212-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33280000 QueuedCount 50000 2016-04-07T04:31:00.38-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33290000 QueuedCount 50000 2016-04-07T04:31:01.655-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:01.865-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33300000 QueuedCount 50000 2016-04-07T04:31:03.297-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33310000 QueuedCount 50000 2016-04-07T04:31:04.62-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33320000 QueuedCount 50000 2016-04-07T04:31:05.675-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33330000 QueuedCount 47966 2016-04-07T04:31:06.682-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33340000 QueuedCount 37966 2016-04-07T04:31:10.081-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33350000 QueuedCount 27966 2016-04-07T04:31:11.105-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33360000 QueuedCount 17966 2016-04-07T04:31:12.232-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33370000 QueuedCount 7966 2016-04-07T04:31:12.955-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":3397192,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":4481116226,"bucket-1:myindex:avg_ts_items_count":28303,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9008881305,"bucket-1:myindex:delete_bytes":565613830,"bucket-1:myindex:disk_size":117004099584,"bucket-1:myindex:flush_queue_size":2336,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":574726992,"bucket-1:myindex:insert_bytes":3481149945,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":382,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":33966958,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":6033042,"bucket-1:myindex:num_flush_queued":33377966,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":571,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1555868610,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"382 225526329261 -9129502006809333575","bucket-1:myindex:timings/storage_del":"13163559 447701723729 -649047485230771281","bucket-1:myindex:timings/storage_get":"33375631 292296680221 1868047685319410633","bucket-1:myindex:timings/storage_handle_open":"382 9164013 235430757527","bucket-1:myindex:timings/storage_info":"1907 17261004 445368013264","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"385 232352778 3700341642750812","bucket-1:myindex:timings/storage_meta_set":"382 495604889 107891164866279445","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"66327116 1561517443495 -1343758864953557609","bucket-1:myindex:timings/storage_snapshot_close":"189 8167068158 371038340218017744","bucket-1:myindex:timings/storage_snapshot_create":"189 73835679370 1550211487045386326","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":254,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"7448 1209047981 221349906698165"} 2016-04-07T04:31:20.046-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 10.823µs CommitTime 4.555046428s TotalFlushTime 38m46.028754613s TotalCommitTime 3m50.081375689s 2016-04-07T04:31:20.046-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46331714, 33165857, 383 committed:true 2016-04-07T04:31:20.046-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46331714, 33165857, 383 committed:true) SnapCreateDur 4.56766856s SnapOpenDur 69.012µs 2016-04-07T04:31:20.245-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33380000 QueuedCount 24631 2016-04-07T04:31:21.78-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33390000 QueuedCount 14631 2016-04-07T04:31:22.931-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33400000 QueuedCount 4631 2016-04-07T04:31:24.05-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.381µs CommitTime 501.234395ms TotalFlushTime 38m49.48007005s TotalCommitTime 3m50.582610084s 2016-04-07T04:31:24.05-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46384250, 33192125, 384 committed:true 2016-04-07T04:31:24.05-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46384250, 33192125, 384 committed:true) SnapCreateDur 512.742493ms SnapOpenDur 33.725µs 2016-04-07T04:31:24.072-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:24.105-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:24.129-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:24.151-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:24.172-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:25.811-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33410000 QueuedCount 47949 2016-04-07T04:31:27.084-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33420000 QueuedCount 37949 2016-04-07T04:31:28.163-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33430000 QueuedCount 27949 2016-04-07T04:31:29.512-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33440000 QueuedCount 17949 2016-04-07T04:31:30.607-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33450000 QueuedCount 7949 2016-04-07T04:31:31.449-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46489110, 33244555, 0 committed:false 2016-04-07T04:31:31.933-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46489110, 33244555, 0 committed:false) SnapCreateDur 20.303µs SnapOpenDur 484.226196ms 2016-04-07T04:31:32.116-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33460000 QueuedCount 1519 2016-04-07T04:31:33.133-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.105µs CommitTime 846.569845ms TotalFlushTime 38m57.13929161s TotalCommitTime 3m51.429179929s 2016-04-07T04:31:33.133-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46496124, 33248062, 385 committed:true 2016-04-07T04:31:33.133-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46496124, 33248062, 385 committed:true) SnapCreateDur 857.90866ms SnapOpenDur 47.784µs 2016-04-07T04:31:33.158-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:33.19-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:33.958-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:34.501-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33470000 QueuedCount 50000 2016-04-07T04:31:36.002-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33480000 QueuedCount 50000 2016-04-07T04:31:37.11-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33490000 QueuedCount 50000 2016-04-07T04:31:37.743-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:38.198-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33500000 QueuedCount 47572 2016-04-07T04:31:39.25-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33510000 QueuedCount 37572 2016-04-07T04:31:44.536-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33520000 QueuedCount 27572 2016-04-07T04:31:45.563-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33530000 QueuedCount 17572 2016-04-07T04:31:46.556-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33540000 QueuedCount 7572 2016-04-07T04:31:48.327-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.98µs CommitTime 927.456257ms TotalFlushTime 39m11.263160493s TotalCommitTime 3m52.356636186s 2016-04-07T04:31:48.327-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46665532, 33332766, 386 committed:true 2016-04-07T04:31:48.327-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46665532, 33332766, 386 committed:true) SnapCreateDur 937.883273ms SnapOpenDur 43.027µs 2016-04-07T04:31:48.529-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33550000 QueuedCount 33059 2016-04-07T04:31:50.793-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33560000 QueuedCount 23059 2016-04-07T04:31:52.025-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33570000 QueuedCount 13059 2016-04-07T04:31:53.097-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33580000 QueuedCount 3059 2016-04-07T04:31:54.105-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.326µs CommitTime 672.401359ms TotalFlushTime 39m16.313980471s TotalCommitTime 3m53.029037545s 2016-04-07T04:31:54.105-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46735378, 33367689, 387 committed:true 2016-04-07T04:31:54.105-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46735378, 33367689, 387 committed:true) SnapCreateDur 682.716962ms SnapOpenDur 40.088µs 2016-04-07T04:31:54.8-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33590000 QueuedCount 4195 2016-04-07T04:31:55.704-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.088µs CommitTime 282.444523ms TotalFlushTime 39m17.58989582s TotalCommitTime 3m53.311482068s 2016-04-07T04:31:55.704-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46757266, 33378633, 388 committed:true 2016-04-07T04:31:55.704-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46757266, 33378633, 388 committed:true) SnapCreateDur 293.592712ms SnapOpenDur 34.894µs 2016-04-07T04:31:55.727-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:55.751-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:55.786-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:31:56.38-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33600000 QueuedCount 50000 2016-04-07T04:31:59.386-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33610000 QueuedCount 50000 2016-04-07T04:32:00.487-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33620000 QueuedCount 50000 2016-04-07T04:32:01.247-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:01.598-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33630000 QueuedCount 48084 2016-04-07T04:32:02.7-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33640000 QueuedCount 38084 2016-04-07T04:32:03.837-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33650000 QueuedCount 28084 2016-04-07T04:32:04.864-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33660000 QueuedCount 18084 2016-04-07T04:32:05.85-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33670000 QueuedCount 8084 2016-04-07T04:32:07.769-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.891µs CommitTime 1.033087192s TotalFlushTime 39m28.486403537s TotalCommitTime 3m54.34456926s 2016-04-07T04:32:07.769-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46922384, 33461192, 389 committed:true 2016-04-07T04:32:07.769-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46922384, 33461192, 389 committed:true) SnapCreateDur 1.044460333s SnapOpenDur 48.954µs 2016-04-07T04:32:09.449-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33680000 QueuedCount 178 2016-04-07T04:32:09.731-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.759µs CommitTime 129.636211ms TotalFlushTime 39m30.283382333s TotalCommitTime 3m54.474205471s 2016-04-07T04:32:09.732-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 46926526, 33463263, 390 committed:true 2016-04-07T04:32:09.732-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 46926526, 33463263, 390 committed:true) SnapCreateDur 140.412543ms SnapOpenDur 45.734µs 2016-04-07T04:32:09.753-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:09.778-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:09.807-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:09.833-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:12.3-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33690000 QueuedCount 50000 2016-04-07T04:32:12.955-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":3029718,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":5180743241,"bucket-1:myindex:avg_ts_items_count":29155,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9067921211,"bucket-1:myindex:delete_bytes":579272737,"bucket-1:myindex:disk_size":120316022784,"bucket-1:myindex:flush_queue_size":50678,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":588605285,"bucket-1:myindex:insert_bytes":3514503152,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":390,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":34347585,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":5652415,"bucket-1:myindex:num_flush_queued":33746117,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":580,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":1962474107,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"390 234474205471 -3383733399805910113","bucket-1:myindex:timings/storage_del":"13478289 460254502830 746097846445713330","bucket-1:myindex:timings/storage_get":"33695440 299663576678 2035230510452303102","bucket-1:myindex:timings/storage_handle_open":"390 9351070 240051667832","bucket-1:myindex:timings/storage_info":"1941 17542466 450495022704","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"393 235559344 3701645410500428","bucket-1:myindex:timings/storage_meta_set":"390 499824609 107893452210158635","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"66956576 1591206722580 8083848323917427742","bucket-1:myindex:timings/storage_snapshot_close":"190 8217979875 373630343145905833","bucket-1:myindex:timings/storage_snapshot_create":"190 74319793508 1784577985656869370","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":228,"memory_quota":31457280000,"memory_used":18874363904,"needs_restart":false,"num_connections":0,"timings/stats_response":"7591 1234423827 225926353333295"} 2016-04-07T04:32:13.482-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33700000 QueuedCount 50000 2016-04-07T04:32:16.952-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:16.953-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33710000 QueuedCount 49939 2016-04-07T04:32:18.031-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33720000 QueuedCount 50000 2016-04-07T04:32:19.066-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33730000 QueuedCount 50000 2016-04-07T04:32:20.193-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33740000 QueuedCount 50000 2016-04-07T04:32:21.259-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33750000 QueuedCount 44522 2016-04-07T04:32:22.281-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33760000 QueuedCount 34522 2016-04-07T04:32:23.256-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33770000 QueuedCount 24522 2016-04-07T04:32:24.249-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33780000 QueuedCount 14522 2016-04-07T04:32:25.336-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33790000 QueuedCount 4522 2016-04-07T04:32:26.893-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.847µs CommitTime 1.136648758s TotalFlushTime 39m46.081474337s TotalCommitTime 3m55.610854229s 2016-04-07T04:32:26.893-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47151668, 33575834, 391 committed:true 2016-04-07T04:32:26.893-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47151668, 33575834, 391 committed:true) SnapCreateDur 1.1489203s SnapOpenDur 66.485µs 2016-04-07T04:32:27.451-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.123µs CommitTime 124.421686ms TotalFlushTime 39m46.474021478s TotalCommitTime 3m55.735275915s 2016-04-07T04:32:27.451-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47158128, 33579064, 392 committed:true 2016-04-07T04:32:27.451-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47158128, 33579064, 392 committed:true) SnapCreateDur 135.585597ms SnapOpenDur 44.069µs 2016-04-07T04:32:27.476-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:27.501-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:27.66-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33800000 QueuedCount 50000 2016-04-07T04:32:27.828-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:28.328-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:28.348-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:28.929-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33810000 QueuedCount 50000 2016-04-07T04:32:30.067-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33820000 QueuedCount 50000 2016-04-07T04:32:31.525-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33830000 QueuedCount 50000 2016-04-07T04:32:39.608-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33840000 QueuedCount 50000 2016-04-07T04:32:40.646-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33850000 QueuedCount 50000 2016-04-07T04:32:41.715-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33860000 QueuedCount 42109 2016-04-07T04:32:42.845-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33870000 QueuedCount 32109 2016-04-07T04:32:45.035-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33880000 QueuedCount 22109 2016-04-07T04:32:48.544-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33890000 QueuedCount 12109 2016-04-07T04:32:49.492-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33900000 QueuedCount 2109 2016-04-07T04:32:49.742-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47363444, 33681722, 0 committed:false 2016-04-07T04:32:50.412-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47363444, 33681722, 0 committed:false) SnapCreateDur 10.348µs SnapOpenDur 669.573399ms 2016-04-07T04:32:51.887-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.361µs CommitTime 1.412802057s TotalFlushTime 40m8.601711668s TotalCommitTime 3m57.148077972s 2016-04-07T04:32:51.887-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47365162, 33682581, 393 committed:true 2016-04-07T04:32:51.887-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47365162, 33682581, 393 committed:true) SnapCreateDur 1.424139811s SnapOpenDur 47.171µs 2016-04-07T04:32:51.916-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:51.947-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:53.459-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33910000 QueuedCount 50000 2016-04-07T04:32:54.588-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33920000 QueuedCount 50000 2016-04-07T04:32:55.228-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:55.675-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33930000 QueuedCount 50000 2016-04-07T04:32:55.841-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:32:56.778-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33940000 QueuedCount 49476 2016-04-07T04:32:57.854-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33950000 QueuedCount 39476 2016-04-07T04:32:59.053-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33960000 QueuedCount 29476 2016-04-07T04:33:00.068-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33970000 QueuedCount 19476 2016-04-07T04:33:01.097-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33980000 QueuedCount 9476 2016-04-07T04:33:03.042-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.21µs CommitTime 943.884431ms TotalFlushTime 40m18.65373994s TotalCommitTime 3m58.091962403s 2016-04-07T04:33:03.042-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47535364, 33767682, 394 committed:true 2016-04-07T04:33:03.042-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47535364, 33767682, 394 committed:true) SnapCreateDur 955.499455ms SnapOpenDur 43.337µs 2016-04-07T04:33:03.12-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 6.426µs CommitTime 36.871175ms TotalFlushTime 40m18.661427023s TotalCommitTime 3m58.128833578s 2016-04-07T04:33:03.12-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47535964, 33767982, 395 committed:true 2016-04-07T04:33:03.121-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47535964, 33767982, 395 committed:true) SnapCreateDur 47.238007ms SnapOpenDur 36.281µs 2016-04-07T04:33:03.147-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:03.148-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 33990000 QueuedCount 15664 2016-04-07T04:33:03.171-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:04.278-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:04.281-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34000000 QueuedCount 49874 2016-04-07T04:33:05.394-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34010000 QueuedCount 50000 2016-04-07T04:33:06.183-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:06.527-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34020000 QueuedCount 50000 2016-04-07T04:33:06.669-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:08.591-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34030000 QueuedCount 50000 2016-04-07T04:33:12.956-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":2692333,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":4813859292,"bucket-1:myindex:avg_ts_items_count":30749,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9130706971,"bucket-1:myindex:delete_bytes":593726757,"bucket-1:myindex:disk_size":122363854848,"bucket-1:myindex:flush_queue_size":50830,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":603291763,"bucket-1:myindex:insert_bytes":3549797895,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":395,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":34690229,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":5309771,"bucket-1:myindex:num_flush_queued":34083654,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":586,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":78138125,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"395 238128833578 812004710097620682","bucket-1:myindex:timings/storage_del":"13810331 476279147396 -399656043987690210","bucket-1:myindex:timings/storage_get":"34032825 309394415209 2331784647952625019","bucket-1:myindex:timings/storage_handle_open":"395 9480471 243468321119","bucket-1:myindex:timings/storage_info":"1963 17731333 454456690253","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"398 237613900 3702497024670244","bucket-1:myindex:timings/storage_meta_set":"395 502609792 107895010756683640","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"67620660 1620189464803 -1374250196531104475","bucket-1:myindex:timings/storage_snapshot_close":"191 8290386641 378873082908484589","bucket-1:myindex:timings/storage_snapshot_create":"191 74989292125 2232806383821782059","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":207,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"7735 1260827017 230918054559051"} 2016-04-07T04:33:13.772-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34040000 QueuedCount 50000 2016-04-07T04:33:14.796-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34050000 QueuedCount 41479 2016-04-07T04:33:15.764-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34060000 QueuedCount 31479 2016-04-07T04:33:16.739-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34070000 QueuedCount 21479 2016-04-07T04:33:17.808-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34080000 QueuedCount 11479 2016-04-07T04:33:18.739-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34090000 QueuedCount 1479 2016-04-07T04:33:18.889-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47736154, 33868077, 0 committed:false 2016-04-07T04:33:19.559-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47736154, 33868077, 0 committed:false) SnapCreateDur 10.707µs SnapOpenDur 670.105904ms 2016-04-07T04:33:21.086-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34100000 QueuedCount 8064 2016-04-07T04:33:23.001-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.289µs CommitTime 1.03393445s TotalFlushTime 40m36.629227013s TotalCommitTime 3m59.162768028s 2016-04-07T04:33:23.001-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47768826, 33884413, 396 committed:true 2016-04-07T04:33:23.001-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47768826, 33884413, 396 committed:true) SnapCreateDur 1.044795139s SnapOpenDur 135.733µs 2016-04-07T04:33:23.027-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:23.049-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:23.07-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:23.092-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:23.116-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:23.213-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34110000 QueuedCount 50000 2016-04-07T04:33:24.474-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34120000 QueuedCount 50000 2016-04-07T04:33:25.519-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34130000 QueuedCount 41409 2016-04-07T04:33:26.542-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34140000 QueuedCount 31409 2016-04-07T04:33:30.41-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34150000 QueuedCount 21409 2016-04-07T04:33:31.437-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34160000 QueuedCount 11409 2016-04-07T04:33:32.426-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34170000 QueuedCount 1409 2016-04-07T04:33:33.28-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47893536, 33946768, 0 committed:false 2016-04-07T04:33:33.901-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47893536, 33946768, 0 committed:false) SnapCreateDur 13.613µs SnapOpenDur 621.056309ms 2016-04-07T04:33:36.226-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34180000 QueuedCount 9999 2016-04-07T04:33:38.45-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 1.865µs CommitTime 947.846705ms TotalFlushTime 40m50.384164091s TotalCommitTime 4m0.110614733s 2016-04-07T04:33:38.451-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47930086, 33965043, 397 committed:true 2016-04-07T04:33:38.451-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47930086, 33965043, 397 committed:true) SnapCreateDur 959.593337ms SnapOpenDur 55.758µs 2016-04-07T04:33:38.471-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34190000 QueuedCount 70 2016-04-07T04:33:38.472-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:38.494-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:38.516-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:38.537-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:38.563-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:39.552-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34200000 QueuedCount 18248 2016-04-07T04:33:41.788-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34210000 QueuedCount 8248 2016-04-07T04:33:42.769-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 47985752, 33992876, 0 committed:false 2016-04-07T04:33:43.068-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 47985752, 33992876, 0 committed:false) SnapCreateDur 19.444µs SnapOpenDur 298.850271ms 2016-04-07T04:33:43.247-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34220000 QueuedCount 14661 2016-04-07T04:33:47.25-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34230000 QueuedCount 4661 2016-04-07T04:33:48.578-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 11.529µs CommitTime 731.304564ms TotalFlushTime 40m59.399830712s TotalCommitTime 4m0.841919297s 2016-04-07T04:33:48.578-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48018046, 34009023, 398 committed:true 2016-04-07T04:33:48.578-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48018046, 34009023, 398 committed:true) SnapCreateDur 743.003967ms SnapOpenDur 38.679µs 2016-04-07T04:33:49.079-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34240000 QueuedCount 5821 2016-04-07T04:33:50.065-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.509µs CommitTime 230.964729ms TotalFlushTime 41m0.615759749s TotalCommitTime 4m1.072884026s 2016-04-07T04:33:50.065-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48040008, 34020004, 399 committed:true 2016-04-07T04:33:50.065-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48040008, 34020004, 399 committed:true) SnapCreateDur 241.606789ms SnapOpenDur 50.809µs 2016-04-07T04:33:50.1-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:50.589-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34250000 QueuedCount 50000 2016-04-07T04:33:52.568-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34260000 QueuedCount 50000 2016-04-07T04:33:55.857-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:33:57.669-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34270000 QueuedCount 50000 2016-04-07T04:33:59.294-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34280000 QueuedCount 50000 2016-04-07T04:34:00.345-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34290000 QueuedCount 43577 2016-04-07T04:34:01.513-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34300000 QueuedCount 33577 2016-04-07T04:34:04.839-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34310000 QueuedCount 23577 2016-04-07T04:34:05.983-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34320000 QueuedCount 13577 2016-04-07T04:34:06.978-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34330000 QueuedCount 3577 2016-04-07T04:34:08.373-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.189µs CommitTime 965.800419ms TotalFlushTime 41m17.789909741s TotalCommitTime 4m2.038684445s 2016-04-07T04:34:08.373-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48212812, 34106406, 400 committed:true 2016-04-07T04:34:08.373-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48212812, 34106406, 400 committed:true) SnapCreateDur 977.209198ms SnapOpenDur 38.944µs 2016-04-07T04:34:10.822-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.636µs CommitTime 254.645022ms TotalFlushTime 41m19.94213379s TotalCommitTime 4m2.293329467s 2016-04-07T04:34:10.822-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48223542, 34111771, 401 committed:true 2016-04-07T04:34:10.822-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48223542, 34111771, 401 committed:true) SnapCreateDur 266.08978ms SnapOpenDur 35.298µs 2016-04-07T04:34:10.859-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:34:10.87-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34340000 QueuedCount 40541 2016-04-07T04:34:11.68-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:34:12.015-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34350000 QueuedCount 50000 2016-04-07T04:34:12.956-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":2375006,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":6687491205,"bucket-1:myindex:avg_ts_items_count":29076,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9189550371,"bucket-1:myindex:delete_bytes":607276014,"bucket-1:myindex:disk_size":125001637888,"bucket-1:myindex:flush_queue_size":50958,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":617056536,"bucket-1:myindex:insert_bytes":3582883290,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":401,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":35016699,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":4983301,"bucket-1:myindex:num_flush_queued":34401109,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":595,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":2449630580,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"401 242293329467 4365204141113009789","bucket-1:myindex:timings/storage_del":"14122698 491304936507 1854008998119062157","bucket-1:myindex:timings/storage_get":"34350152 316579379689 2506898467098802359","bucket-1:myindex:timings/storage_handle_open":"401 9718954 260327774232","bucket-1:myindex:timings/storage_info":"1993 18030097 462202535439","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"404 240015375 3703463187841503","bucket-1:myindex:timings/storage_meta_set":"401 505791869 107896731403941623","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"68245396 1650520329159 -7956480465879469783","bucket-1:myindex:timings/storage_snapshot_close":"194 8466613241 389921761653352283","bucket-1:myindex:timings/storage_snapshot_create":"194 76579034594 3156591863864841552","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":0,"bucket-1:ts_queue_size":184,"memory_quota":31457280000,"memory_used":18874368000,"needs_restart":false,"num_connections":0,"timings/stats_response":"7879 1287409878 235910866779284"} 2016-04-07T04:34:14.462-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34360000 QueuedCount 50000 2016-04-07T04:34:15.622-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34370000 QueuedCount 50000 2016-04-07T04:34:16.581-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:34:16.736-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34380000 QueuedCount 49999 2016-04-07T04:34:17.876-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34390000 QueuedCount 50000 2016-04-07T04:34:19.021-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34400000 QueuedCount 50000 2016-04-07T04:34:23.984-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34410000 QueuedCount 47622 2016-04-07T04:34:25.068-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34420000 QueuedCount 37622 2016-04-07T04:34:26.155-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34430000 QueuedCount 27622 2016-04-07T04:34:27.176-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34440000 QueuedCount 17622 2016-04-07T04:34:28.233-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34450000 QueuedCount 7622 2016-04-07T04:34:30.255-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.08µs CommitTime 1.171545741s TotalFlushTime 41m37.985836666s TotalCommitTime 4m3.464875208s 2016-04-07T04:34:30.255-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48456776, 34228388, 402 committed:true 2016-04-07T04:34:30.255-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48456776, 34228388, 402 committed:true) SnapCreateDur 1.182720045s SnapOpenDur 58.58µs 2016-04-07T04:34:30.915-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34460000 QueuedCount 28132 2016-04-07T04:34:32.116-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34470000 QueuedCount 18132 2016-04-07T04:34:33.414-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34480000 QueuedCount 8132 2016-04-07T04:34:35.649-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 7.095µs CommitTime 672.379314ms TotalFlushTime 41m42.652541827s TotalCommitTime 4m4.137254522s 2016-04-07T04:34:35.649-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48516780, 34258390, 403 committed:true 2016-04-07T04:34:35.649-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48516780, 34258390, 403 committed:true) SnapCreateDur 683.103431ms SnapOpenDur 31.43µs 2016-04-07T04:34:35.85-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34490000 QueuedCount 10744 2016-04-07T04:34:37.038-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34500000 QueuedCount 744 2016-04-07T04:34:37.477-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.915µs CommitTime 271.121148ms TotalFlushTime 41m44.161162333s TotalCommitTime 4m4.40837567s 2016-04-07T04:34:37.477-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48541550, 34270775, 404 committed:true 2016-04-07T04:34:37.477-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48541550, 34270775, 404 committed:true) SnapCreateDur 282.400349ms SnapOpenDur 38.681µs 2016-04-07T04:34:37.497-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:34:37.526-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:34:41.193-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34510000 QueuedCount 50000 2016-04-07T04:34:41.331-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:34:42.352-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34520000 QueuedCount 50000 2016-04-07T04:34:43.615-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34530000 QueuedCount 48886 2016-04-07T04:34:44.676-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34540000 QueuedCount 38886 2016-04-07T04:34:46.116-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34550000 QueuedCount 28886 2016-04-07T04:34:47.156-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34560000 QueuedCount 18886 2016-04-07T04:34:48.184-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34570000 QueuedCount 8886 2016-04-07T04:34:55.336-07:00 [Info] CompactionDaemon: Finished compacting index instance:9366861204008862690 2016-04-07T04:34:59.259-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 4.098µs CommitTime 609.993763ms TotalFlushTime 42m5.19573718s TotalCommitTime 4m5.018369433s 2016-04-07T04:34:59.259-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48695376, 34347688, 405 committed:true 2016-04-07T04:34:59.259-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48695376, 34347688, 405 committed:true) SnapCreateDur 620.536028ms SnapOpenDur 36.089µs 2016-04-07T04:34:59.62-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34580000 QueuedCount 3991 2016-04-07T04:35:00.151-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 9.427µs CommitTime 168.974987ms TotalFlushTime 42m5.882750073s TotalCommitTime 4m5.18734442s 2016-04-07T04:35:00.152-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48705440, 34352720, 406 committed:true 2016-04-07T04:35:07.413-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48705440, 34352720, 406 committed:true) SnapCreateDur 179.777377ms SnapOpenDur 7.261488983s 2016-04-07T04:35:07.627-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.246µs CommitTime 51.279378ms TotalFlushTime 42m6.006527078s TotalCommitTime 4m5.238623798s 2016-04-07T04:35:07.627-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48708548, 34354274, 407 committed:true 2016-04-07T04:35:07.627-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48708548, 34354274, 407 committed:true) SnapCreateDur 62.127108ms SnapOpenDur 32.363µs 2016-04-07T04:35:07.649-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:07.67-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:07.691-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:07.716-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:07.742-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:08.015-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34590000 QueuedCount 23670 2016-04-07T04:35:08.757-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34600000 QueuedCount 13670 2016-04-07T04:35:09.47-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34610000 QueuedCount 3670 2016-04-07T04:35:09.772-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 48763818, 34381909, 0 committed:false 2016-04-07T04:35:10.006-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 48763818, 34381909, 0 committed:false) SnapCreateDur 13.853µs SnapOpenDur 233.618975ms 2016-04-07T04:35:10.027-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:10.048-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:10.07-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:10.107-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:10.488-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34620000 QueuedCount 50000 2016-04-07T04:35:11.249-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34630000 QueuedCount 50000 2016-04-07T04:35:11.455-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:12.111-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34640000 QueuedCount 50000 2016-04-07T04:35:12.879-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34650000 QueuedCount 50000 2016-04-07T04:35:12.956-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":2074302,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3968953105,"bucket-1:myindex:avg_ts_items_count":21992,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9167224004,"bucket-1:myindex:delete_bytes":619890881,"bucket-1:myindex:disk_size":117035524096,"bucket-1:myindex:flush_queue_size":50068,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":629879437,"bucket-1:myindex:insert_bytes":3613687035,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":407,"bucket-1:myindex:num_compactions":4,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":35303774,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":4696226,"bucket-1:myindex:num_flush_queued":34700923,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":602,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":2378610371,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"407 245238623798 6666598694894814592","bucket-1:myindex:timings/storage_del":"14418516 504888955280 -8169089007561963652","bucket-1:myindex:timings/storage_get":"34650856 321960635775 2583111766426703439","bucket-1:myindex:timings/storage_handle_open":"407 7271276745 -2611440938478942841","bucket-1:myindex:timings/storage_info":"2019 18396953 485992841029","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"410 242300809 3704343194019967","bucket-1:myindex:timings/storage_meta_set":"407 508448649 107897915544439013","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"68837030 1681121353277 5406824218347448081","bucket-1:myindex:timings/storage_snapshot_close":"194 8466613241 389921761653352283","bucket-1:myindex:timings/storage_snapshot_create":"195 76812580432 3211135522311963796","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":2,"bucket-1:ts_queue_size":163,"memory_quota":31457280000,"memory_used":1814990848,"needs_restart":false,"num_connections":0,"timings/stats_response":"8023 1314990665 242663867104581"} 2016-04-07T04:35:13.109-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:13.641-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34660000 QueuedCount 50000 2016-04-07T04:35:14.399-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34670000 QueuedCount 50000 2016-04-07T04:35:14.933-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:15.215-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34680000 QueuedCount 50000 2016-04-07T04:35:15.965-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34690000 QueuedCount 50000 2016-04-07T04:35:16.725-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34700000 QueuedCount 50000 2016-04-07T04:35:17.473-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34710000 QueuedCount 50000 2016-04-07T04:35:18.239-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34720000 QueuedCount 50000 2016-04-07T04:35:19.082-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34730000 QueuedCount 50000 2016-04-07T04:35:19.87-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34740000 QueuedCount 49997 2016-04-07T04:35:20.097-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:20.613-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34750000 QueuedCount 50000 2016-04-07T04:35:21.346-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34760000 QueuedCount 50000 2016-04-07T04:35:21.349-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:21.37-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:21.391-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:22.082-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34770000 QueuedCount 50000 2016-04-07T04:35:22.905-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34780000 QueuedCount 50000 2016-04-07T04:35:23.633-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34790000 QueuedCount 50000 2016-04-07T04:35:24.25-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:24.352-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34800000 QueuedCount 50000 2016-04-07T04:35:24.759-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:24.779-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:24.801-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:24.821-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:25.072-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34810000 QueuedCount 50000 2016-04-07T04:35:25.336-07:00 [Info] CompactionDaemon: Checking fragmentation, IndexInstId: 9366861204008862690 Data:9197874754, Disk:117035524096, ExtraSnapshotData:0, Fragmentation:92.14095478697962% 2016-04-07T04:35:25.336-07:00 [Info] CompactionDaemon: Compacting index instance:9366861204008862690 2016-04-07T04:35:25.398-07:00 [Info] ForestDBSlice::Compact Compacting upto SeqNum 48516780. Slice Id 0, IndexInstId 9366861204008862690, IndexDefnId 9366861204008862690 2016-04-07T04:35:27.02-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34820000 QueuedCount 50000 2016-04-07T04:35:27.985-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34830000 QueuedCount 50000 2016-04-07T04:35:28.256-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:28.866-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34840000 QueuedCount 50000 2016-04-07T04:35:30.437-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34850000 QueuedCount 50000 2016-04-07T04:35:31.286-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34860000 QueuedCount 50000 2016-04-07T04:35:31.932-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:32.064-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34870000 QueuedCount 50000 2016-04-07T04:35:32.958-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34880000 QueuedCount 50000 2016-04-07T04:35:33.779-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34890000 QueuedCount 50000 2016-04-07T04:35:34.338-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:34.577-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34900000 QueuedCount 50000 2016-04-07T04:35:35.389-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34910000 QueuedCount 50000 2016-04-07T04:35:35.393-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:35.489-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:35.85-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:36.211-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34920000 QueuedCount 50000 2016-04-07T04:35:37.079-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34930000 QueuedCount 50000 2016-04-07T04:35:37.885-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34940000 QueuedCount 50000 2016-04-07T04:35:38.673-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34950000 QueuedCount 50000 2016-04-07T04:35:40.246-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34960000 QueuedCount 50000 2016-04-07T04:35:40.481-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:41.111-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34970000 QueuedCount 50000 2016-04-07T04:35:41.869-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34980000 QueuedCount 50000 2016-04-07T04:35:42.639-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 34990000 QueuedCount 50000 2016-04-07T04:35:43.413-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35000000 QueuedCount 50000 2016-04-07T04:35:43.509-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:44.233-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35010000 QueuedCount 50000 2016-04-07T04:35:45.162-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35020000 QueuedCount 50000 2016-04-07T04:35:45.939-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35030000 QueuedCount 50000 2016-04-07T04:35:46.164-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:46.385-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:46.73-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35040000 QueuedCount 50000 2016-04-07T04:35:47.619-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35050000 QueuedCount 50000 2016-04-07T04:35:48.535-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:48.542-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35060000 QueuedCount 49667 2016-04-07T04:35:49.46-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35070000 QueuedCount 50000 2016-04-07T04:35:50.379-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35080000 QueuedCount 50000 2016-04-07T04:35:51.257-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35090000 QueuedCount 50000 2016-04-07T04:35:51.701-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:51.779-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:51.998-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35100000 QueuedCount 46322 2016-04-07T04:35:52.815-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35110000 QueuedCount 50000 2016-04-07T04:35:53.798-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35120000 QueuedCount 50000 2016-04-07T04:35:54.57-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35130000 QueuedCount 50000 2016-04-07T04:35:54.818-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:55.351-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35140000 QueuedCount 50000 2016-04-07T04:35:56.152-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35150000 QueuedCount 50000 2016-04-07T04:35:56.911-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35160000 QueuedCount 50000 2016-04-07T04:35:57.63-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:57.722-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35170000 QueuedCount 50000 2016-04-07T04:35:58.51-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35180000 QueuedCount 50000 2016-04-07T04:35:58.96-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:59.006-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:35:59.284-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35190000 QueuedCount 50000 2016-04-07T04:36:00.032-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35200000 QueuedCount 50000 2016-04-07T04:36:00.874-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35210000 QueuedCount 50000 2016-04-07T04:36:01.33-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:01.66-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35220000 QueuedCount 50000 2016-04-07T04:36:02.364-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35230000 QueuedCount 50000 2016-04-07T04:36:02.577-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:02.599-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:02.883-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:03.082-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35240000 QueuedCount 50000 2016-04-07T04:36:03.778-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35250000 QueuedCount 50000 2016-04-07T04:36:04.559-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35260000 QueuedCount 50000 2016-04-07T04:36:05.258-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35270000 QueuedCount 50000 2016-04-07T04:36:05.945-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35280000 QueuedCount 50000 2016-04-07T04:36:06.675-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35290000 QueuedCount 49962 2016-04-07T04:36:06.919-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:07.418-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35300000 QueuedCount 50000 2016-04-07T04:36:08.18-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35310000 QueuedCount 50000 2016-04-07T04:36:08.901-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35320000 QueuedCount 50000 2016-04-07T04:36:09.411-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:09.606-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35330000 QueuedCount 50000 2016-04-07T04:36:10.045-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:10.344-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35340000 QueuedCount 50000 2016-04-07T04:36:11.092-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35350000 QueuedCount 50000 2016-04-07T04:36:11.094-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:11.872-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35360000 QueuedCount 50000 2016-04-07T04:36:12.607-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35370000 QueuedCount 50000 2016-04-07T04:36:12.956-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":1350579,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3968953105,"bucket-1:myindex:avg_ts_items_count":21992,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9287539954,"bucket-1:myindex:delete_bytes":647832496,"bucket-1:myindex:disk_size":117907763200,"bucket-1:myindex:flush_queue_size":50087,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":658265930,"bucket-1:myindex:insert_bytes":3681916560,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":407,"bucket-1:myindex:num_compactions":5,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":36337193,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":3662807,"bucket-1:myindex:num_flush_queued":35424664,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":602,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":2378610371,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"407 245238623798 6666598694894814592","bucket-1:myindex:timings/storage_del":"15130930 520860549488 -7011749978246230028","bucket-1:myindex:timings/storage_get":"35374578 330935385514 2586487565036220440","bucket-1:myindex:timings/storage_handle_open":"407 7271276745 -2611440938478942841","bucket-1:myindex:timings/storage_info":"2019 18396953 485992841029","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"411 250289867 3768168241747331","bucket-1:myindex:timings/storage_meta_set":"407 508448649 107897915544439013","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"70261860 1713841398858 -8840146445099522410","bucket-1:myindex:timings/storage_snapshot_close":"194 8466613241 389921761653352283","bucket-1:myindex:timings/storage_snapshot_create":"195 76812580432 3211135522311963796","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":38,"bucket-1:ts_queue_size":121,"memory_quota":31457280000,"memory_used":6494867456,"needs_restart":false,"num_connections":0,"timings/stats_response":"8165 1342266770 248703405466460"} 2016-04-07T04:36:13.375-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:13.376-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35380000 QueuedCount 49964 2016-04-07T04:36:14.005-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:14.027-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:14.115-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35390000 QueuedCount 50000 2016-04-07T04:36:14.844-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35400000 QueuedCount 50000 2016-04-07T04:36:15.624-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35410000 QueuedCount 50000 2016-04-07T04:36:16.326-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35420000 QueuedCount 50000 2016-04-07T04:36:16.75-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:17.052-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35430000 QueuedCount 50000 2016-04-07T04:36:17.761-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35440000 QueuedCount 50000 2016-04-07T04:36:18.533-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35450000 QueuedCount 50000 2016-04-07T04:36:19.054-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:19.075-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:19.095-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:19.115-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:19.139-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:19.16-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:19.221-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35460000 QueuedCount 50000 2016-04-07T04:36:19.95-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35470000 QueuedCount 50000 2016-04-07T04:36:20.686-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35480000 QueuedCount 50000 2016-04-07T04:36:20.983-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:21.388-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35490000 QueuedCount 50000 2016-04-07T04:36:22.172-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35500000 QueuedCount 50000 2016-04-07T04:36:22.92-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35510000 QueuedCount 50000 2016-04-07T04:36:22.938-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:23.613-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35520000 QueuedCount 50000 2016-04-07T04:36:24.324-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35530000 QueuedCount 50000 2016-04-07T04:36:25.032-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35540000 QueuedCount 50000 2016-04-07T04:36:25.688-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:25.849-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35550000 QueuedCount 50000 2016-04-07T04:36:26.645-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35560000 QueuedCount 50000 2016-04-07T04:36:26.652-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:26.673-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:26.693-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:26.713-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:27.38-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35570000 QueuedCount 50000 2016-04-07T04:36:27.596-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:28.078-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35580000 QueuedCount 50000 2016-04-07T04:36:28.791-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35590000 QueuedCount 50000 2016-04-07T04:36:29.615-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35600000 QueuedCount 50000 2016-04-07T04:36:30.131-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:30.329-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35610000 QueuedCount 50000 2016-04-07T04:36:31.011-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35620000 QueuedCount 50000 2016-04-07T04:36:31.701-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35630000 QueuedCount 50000 2016-04-07T04:36:31.787-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:32.058-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:32.396-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35640000 QueuedCount 50000 2016-04-07T04:36:33.153-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35650000 QueuedCount 50000 2016-04-07T04:36:33.906-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35660000 QueuedCount 50000 2016-04-07T04:36:34.219-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:34.432-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:34.63-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35670000 QueuedCount 50000 2016-04-07T04:36:35.364-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35680000 QueuedCount 50000 2016-04-07T04:36:36.073-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35690000 QueuedCount 50000 2016-04-07T04:36:36.814-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35700000 QueuedCount 50000 2016-04-07T04:36:37.533-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35710000 QueuedCount 50000 2016-04-07T04:36:37.827-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:38.232-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35720000 QueuedCount 50000 2016-04-07T04:36:38.917-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35730000 QueuedCount 50000 2016-04-07T04:36:39.63-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35740000 QueuedCount 50000 2016-04-07T04:36:40.387-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35750000 QueuedCount 50000 2016-04-07T04:36:40.706-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:40.809-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:41.104-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35760000 QueuedCount 50000 2016-04-07T04:36:41.793-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35770000 QueuedCount 50000 2016-04-07T04:36:42.472-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35780000 QueuedCount 50000 2016-04-07T04:36:43.244-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35790000 QueuedCount 50000 2016-04-07T04:36:43.839-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:43.94-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35800000 QueuedCount 50000 2016-04-07T04:36:43.953-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:43.974-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:43.995-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:44.642-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35810000 QueuedCount 50000 2016-04-07T04:36:45.335-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35820000 QueuedCount 50000 2016-04-07T04:36:46.017-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35830000 QueuedCount 50000 2016-04-07T04:36:46.698-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:46.773-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35840000 QueuedCount 50000 2016-04-07T04:36:47.47-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35850000 QueuedCount 50000 2016-04-07T04:36:48.164-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35860000 QueuedCount 50000 2016-04-07T04:36:48.847-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35870000 QueuedCount 50000 2016-04-07T04:36:49.25-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:49.546-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35880000 QueuedCount 50000 2016-04-07T04:36:50.323-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35890000 QueuedCount 50000 2016-04-07T04:36:51.011-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35900000 QueuedCount 50000 2016-04-07T04:36:51.125-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:51.705-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35910000 QueuedCount 50000 2016-04-07T04:36:52.414-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35920000 QueuedCount 50000 2016-04-07T04:36:52.521-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:52.541-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:52.562-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:52.582-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:52.817-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:53.114-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:53.126-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35930000 QueuedCount 49398 2016-04-07T04:36:53.905-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35940000 QueuedCount 50000 2016-04-07T04:36:54.627-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35950000 QueuedCount 50000 2016-04-07T04:36:55.338-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35960000 QueuedCount 50000 2016-04-07T04:36:55.555-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:56.073-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35970000 QueuedCount 50000 2016-04-07T04:36:56.807-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35980000 QueuedCount 50000 2016-04-07T04:36:57.576-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 35990000 QueuedCount 50000 2016-04-07T04:36:58.292-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36000000 QueuedCount 50000 2016-04-07T04:36:59.029-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36010000 QueuedCount 50000 2016-04-07T04:36:59.428-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:36:59.723-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36020000 QueuedCount 50000 2016-04-07T04:37:00.427-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36030000 QueuedCount 50000 2016-04-07T04:37:01.199-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36040000 QueuedCount 50000 2016-04-07T04:37:01.63-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:01.904-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36050000 QueuedCount 46394 2016-04-07T04:37:02.206-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:02.674-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36060000 QueuedCount 50000 2016-04-07T04:37:03.624-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36070000 QueuedCount 50000 2016-04-07T04:37:04.297-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:04.599-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36080000 QueuedCount 50000 2016-04-07T04:37:05.418-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36090000 QueuedCount 50000 2016-04-07T04:37:05.822-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:05.844-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:06.116-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36100000 QueuedCount 50000 2016-04-07T04:37:06.213-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:06.801-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36110000 QueuedCount 50000 2016-04-07T04:37:07.183-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:07.203-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:07.271-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:07.496-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36120000 QueuedCount 50000 2016-04-07T04:37:07.986-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:08.274-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36130000 QueuedCount 50000 2016-04-07T04:37:09.067-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36140000 QueuedCount 50000 2016-04-07T04:37:09.804-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36150000 QueuedCount 50000 2016-04-07T04:37:10.514-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36160000 QueuedCount 50000 2016-04-07T04:37:11.209-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36170000 QueuedCount 50000 2016-04-07T04:37:11.516-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:12.012-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36180000 QueuedCount 50000 2016-04-07T04:37:12.714-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36190000 QueuedCount 50000 2016-04-07T04:37:12.82-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:12.84-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:12.86-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:12.881-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:12.957-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":531306,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":3968953105,"bucket-1:myindex:avg_ts_items_count":21992,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9438644252,"bucket-1:myindex:delete_bytes":682600447,"bucket-1:myindex:disk_size":117907763200,"bucket-1:myindex:flush_queue_size":50269,"bucket-1:myindex:frag_percent":91,"bucket-1:myindex:get_bytes":693592795,"bucket-1:myindex:insert_bytes":3766814940,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":407,"bucket-1:myindex:num_compactions":5,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":38501720,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":1498280,"bucket-1:myindex:num_flush_queued":36244120,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":602,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":2378610371,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"407 245238623798 6666598694894814592","bucket-1:myindex:timings/storage_del":"15937168 537124155355 -6008411198169082413","bucket-1:myindex:timings/storage_get":"36193852 340877258996 2592599204953178798","bucket-1:myindex:timings/storage_handle_open":"407 7271276745 -2611440938478942841","bucket-1:myindex:timings/storage_info":"2019 18396953 485992841029","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"411 250289867 3768168241747331","bucket-1:myindex:timings/storage_meta_set":"407 508448649 107897915544439013","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"71874334 1745126759856 -6856662718322808614","bucket-1:myindex:timings/storage_snapshot_close":"194 8466613241 389921761653352283","bucket-1:myindex:timings/storage_snapshot_create":"195 76812580432 3211135522311963796","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":94,"bucket-1:ts_queue_size":60,"memory_quota":31457280000,"memory_used":9292447744,"needs_restart":false,"num_connections":0,"timings/stats_response":"8309 1369420799 253970121707977"} 2016-04-07T04:37:13.429-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36200000 QueuedCount 50000 2016-04-07T04:37:14.139-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36210000 QueuedCount 50000 2016-04-07T04:37:14.519-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:14.86-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36220000 QueuedCount 50000 2016-04-07T04:37:14.864-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:14.885-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:15.627-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36230000 QueuedCount 50000 2016-04-07T04:37:16.313-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36240000 QueuedCount 50000 2016-04-07T04:37:16.696-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:16.987-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36250000 QueuedCount 50000 2016-04-07T04:37:17.69-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36260000 QueuedCount 50000 2016-04-07T04:37:18.398-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36270000 QueuedCount 50000 2016-04-07T04:37:19.156-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36280000 QueuedCount 50000 2016-04-07T04:37:19.781-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:19.859-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36290000 QueuedCount 50000 2016-04-07T04:37:20.55-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36300000 QueuedCount 50000 2016-04-07T04:37:21.244-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36310000 QueuedCount 50000 2016-04-07T04:37:21.251-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:21.271-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:21.329-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:21.782-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:21.957-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:21.975-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36320000 QueuedCount 49012 2016-04-07T04:37:22.73-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36330000 QueuedCount 50000 2016-04-07T04:37:23.437-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36340000 QueuedCount 50000 2016-04-07T04:37:24.13-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36350000 QueuedCount 50000 2016-04-07T04:37:24.868-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36360000 QueuedCount 50000 2016-04-07T04:37:25.093-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:25.67-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36370000 QueuedCount 50000 2016-04-07T04:37:26.06-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:26.081-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:26.343-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36380000 QueuedCount 50000 2016-04-07T04:37:27.039-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36390000 QueuedCount 50000 2016-04-07T04:37:27.342-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:27.363-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:27.383-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:27.403-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:27.424-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:27.444-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:27.464-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:27.722-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36400000 QueuedCount 50000 2016-04-07T04:37:27.728-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:27.8-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:28.031-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:28.441-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36410000 QueuedCount 50000 2016-04-07T04:37:29.205-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36420000 QueuedCount 50000 2016-04-07T04:37:29.898-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36430000 QueuedCount 50000 2016-04-07T04:37:30.584-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36440000 QueuedCount 50000 2016-04-07T04:37:31.161-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:31.263-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36450000 QueuedCount 50000 2016-04-07T04:37:31.973-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36460000 QueuedCount 50000 2016-04-07T04:37:32.727-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36470000 QueuedCount 50000 2016-04-07T04:37:33.428-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36480000 QueuedCount 50000 2016-04-07T04:37:33.537-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:34.134-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36490000 QueuedCount 50000 2016-04-07T04:37:34.835-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36500000 QueuedCount 50000 2016-04-07T04:37:35.539-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36510000 QueuedCount 50000 2016-04-07T04:37:35.738-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:36.317-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36520000 QueuedCount 50000 2016-04-07T04:37:37.016-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36530000 QueuedCount 50000 2016-04-07T04:37:37.408-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.428-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.45-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.47-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.49-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.511-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.531-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.551-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.572-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.592-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.614-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:37.71-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36540000 QueuedCount 50000 2016-04-07T04:37:38.429-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36550000 QueuedCount 50000 2016-04-07T04:37:39.119-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36560000 QueuedCount 50000 2016-04-07T04:37:39.911-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36570000 QueuedCount 50000 2016-04-07T04:37:40.63-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36580000 QueuedCount 50000 2016-04-07T04:37:40.724-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:41.309-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36590000 QueuedCount 50000 2016-04-07T04:37:42.005-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36600000 QueuedCount 50000 2016-04-07T04:37:42.693-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36610000 QueuedCount 50000 2016-04-07T04:37:43.456-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36620000 QueuedCount 50000 2016-04-07T04:37:43.946-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:44.155-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36630000 QueuedCount 50000 2016-04-07T04:37:44.848-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36640000 QueuedCount 50000 2016-04-07T04:37:45.535-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36650000 QueuedCount 50000 2016-04-07T04:37:46.223-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36660000 QueuedCount 50000 2016-04-07T04:37:46.302-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:46.406-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:46.487-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:46.87-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:46.971-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36670000 QueuedCount 50000 2016-04-07T04:37:47.173-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:47.195-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:47.68-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36680000 QueuedCount 50000 2016-04-07T04:37:48.4-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36690000 QueuedCount 50000 2016-04-07T04:37:49.095-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36700000 QueuedCount 50000 2016-04-07T04:37:49.293-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:49.869-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:49.87-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36710000 QueuedCount 49995 2016-04-07T04:37:49.891-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:49.912-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:49.975-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:50.25-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:50.271-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:50.531-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36720000 QueuedCount 45665 2016-04-07T04:37:51.005-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:51.027-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:51.05-07:00 [Info] StorageMgr::handleCreateSnapshot Skip Snapshot For MAINT_STREAM bucket-1 SnapType NO_SNAP 2016-04-07T04:37:51.168-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36730000 QueuedCount 45158 2016-04-07T04:37:51.793-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36740000 QueuedCount 35158 2016-04-07T04:37:52.421-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36750000 QueuedCount 25158 2016-04-07T04:37:53.107-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36760000 QueuedCount 15158 2016-04-07T04:37:53.718-07:00 [Info] logWriterStat:: 9366861204008862690 FlushedCount 36770000 QueuedCount 5158 2016-04-07T04:37:56.282-07:00 [Info] ForestDBSlice::Commit SliceId 0 IndexInstId 9366861204008862690 FlushTime 3.352µs CommitTime 2.25021867s TotalFlushTime 44m47.045431093s TotalCommitTime 4m7.488842468s 2016-04-07T04:37:56.282-07:00 [Info] ForestDBSlice::OpenSnapshot SliceId 0 IndexInstId 9366861204008862690 Creating New Snapshot SnapshotInfo: seqnos: 53018614, 36509307, 408 committed:true 2016-04-07T04:37:56.282-07:00 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9366861204008862690 PartitionId: 0 SliceId: 0 Crc64: 6632717683445782364 (SnapshotInfo: seqnos: 53018614, 36509307, 408 committed:true) SnapCreateDur 2.260372323s SnapOpenDur 57.95µs 2016-04-07T04:38:12.957-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":85122636702,"bucket-1:myindex:avg_ts_items_count":1091740,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9561519743,"bucket-1:myindex:delete_bytes":709900201,"bucket-1:myindex:disk_size":119686078464,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":721331794,"bucket-1:myindex:insert_bytes":3833477235,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":408,"bucket-1:myindex:num_compactions":5,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":40000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":36775158,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":603,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":166276320300,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"408 247488842468 -6716661315998168124","bucket-1:myindex:timings/storage_del":"16509307 548294690446 -5338774034469764108","bucket-1:myindex:timings/storage_get":"36775158 347467745772 2592724130857847910","bucket-1:myindex:timings/storage_handle_open":"408 7271318701 -2611440936718636905","bucket-1:myindex:timings/storage_info":"2023 18430311 486447162275","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"412 250615813 3768274482542247","bucket-1:myindex:timings/storage_meta_set":"408 508835340 107898065074368494","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"73018614 1766840442396 -5511590561845827484","bucket-1:myindex:timings/storage_snapshot_close":"195 8500634102 391079180636533604","bucket-1:myindex:timings/storage_snapshot_create":"195 76812580432 3211135522311963796","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":148,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":11522912256,"needs_restart":false,"num_connections":0,"timings/stats_response":"8438 1393206930 258526329956888"} 2016-04-07T04:39:12.957-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":85122636702,"bucket-1:myindex:avg_ts_items_count":1091740,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9561519743,"bucket-1:myindex:delete_bytes":709900201,"bucket-1:myindex:disk_size":119686078464,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":721331794,"bucket-1:myindex:insert_bytes":3833477235,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":408,"bucket-1:myindex:num_compactions":5,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":40000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":36775158,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":603,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":166276320300,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"408 247488842468 -6716661315998168124","bucket-1:myindex:timings/storage_del":"16509307 548294690446 -5338774034469764108","bucket-1:myindex:timings/storage_get":"36775158 347467745772 2592724130857847910","bucket-1:myindex:timings/storage_handle_open":"408 7271318701 -2611440936718636905","bucket-1:myindex:timings/storage_info":"2023 18430311 486447162275","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"412 250615813 3768274482542247","bucket-1:myindex:timings/storage_meta_set":"408 508835340 107898065074368494","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"73018614 1766840442396 -5511590561845827484","bucket-1:myindex:timings/storage_snapshot_close":"195 8500634102 391079180636533604","bucket-1:myindex:timings/storage_snapshot_create":"195 76812580432 3211135522311963796","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":148,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":12945424384,"needs_restart":false,"num_connections":0,"timings/stats_response":"8510 1406277145 260975610763347"} 2016-04-07T04:40:12.957-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":85122636702,"bucket-1:myindex:avg_ts_items_count":1091740,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9561519743,"bucket-1:myindex:delete_bytes":709900201,"bucket-1:myindex:disk_size":119686078464,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":721331794,"bucket-1:myindex:insert_bytes":3833477235,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":408,"bucket-1:myindex:num_compactions":5,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":40000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":36775158,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":603,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":166276320300,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"408 247488842468 -6716661315998168124","bucket-1:myindex:timings/storage_del":"16509307 548294690446 -5338774034469764108","bucket-1:myindex:timings/storage_get":"36775158 347467745772 2592724130857847910","bucket-1:myindex:timings/storage_handle_open":"408 7271318701 -2611440936718636905","bucket-1:myindex:timings/storage_info":"2023 18430311 486447162275","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"412 250615813 3768274482542247","bucket-1:myindex:timings/storage_meta_set":"408 508835340 107898065074368494","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"73018614 1766840442396 -5511590561845827484","bucket-1:myindex:timings/storage_snapshot_close":"195 8500634102 391079180636533604","bucket-1:myindex:timings/storage_snapshot_create":"195 76812580432 3211135522311963796","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":148,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":14381604864,"needs_restart":false,"num_connections":0,"timings/stats_response":"8582 1419916046 263609502627118"} 2016-04-07T04:41:12.958-07:00 [Info] PeriodicStats = {"bucket-1:mutation_queue_size":0,"bucket-1:myindex:avg_scan_latency":0,"bucket-1:myindex:avg_scan_wait_latency":0,"bucket-1:myindex:avg_ts_interval":85122636702,"bucket-1:myindex:avg_ts_items_count":1091740,"bucket-1:myindex:build_progress":100,"bucket-1:myindex:data_size":9561519743,"bucket-1:myindex:delete_bytes":709900201,"bucket-1:myindex:disk_size":119686078464,"bucket-1:myindex:flush_queue_size":0,"bucket-1:myindex:frag_percent":92,"bucket-1:myindex:get_bytes":721331794,"bucket-1:myindex:insert_bytes":3833477235,"bucket-1:myindex:items_count":20000000,"bucket-1:myindex:num_commits":408,"bucket-1:myindex:num_compactions":5,"bucket-1:myindex:num_completed_requests":0,"bucket-1:myindex:num_docs_indexed":40000000,"bucket-1:myindex:num_docs_pending":0,"bucket-1:myindex:num_docs_queued":0,"bucket-1:myindex:num_flush_queued":36775158,"bucket-1:myindex:num_last_snapshot_reply":0,"bucket-1:myindex:num_requests":0,"bucket-1:myindex:num_rows_returned":0,"bucket-1:myindex:num_snapshot_waiters":0,"bucket-1:myindex:num_snapshots":603,"bucket-1:myindex:scan_bytes_read":0,"bucket-1:myindex:scan_wait_duration":0,"bucket-1:myindex:since_last_snapshot":166276320300,"bucket-1:myindex:timings/storage_clone_handle":"0 0 0","bucket-1:myindex:timings/storage_commit":"408 247488842468 -6716661315998168124","bucket-1:myindex:timings/storage_del":"16509307 548294690446 -5338774034469764108","bucket-1:myindex:timings/storage_get":"36775158 347467745772 2592724130857847910","bucket-1:myindex:timings/storage_handle_open":"408 7271318701 -2611440936718636905","bucket-1:myindex:timings/storage_info":"2023 18430311 486447162275","bucket-1:myindex:timings/storage_iterator_next":"0 0 0","bucket-1:myindex:timings/storage_meta_get":"412 250615813 3768274482542247","bucket-1:myindex:timings/storage_meta_set":"408 508835340 107898065074368494","bucket-1:myindex:timings/storage_new_iterator":"0 0 0","bucket-1:myindex:timings/storage_set":"73018614 1766840442396 -5511590561845827484","bucket-1:myindex:timings/storage_snapshot_close":"195 8500634102 391079180636533604","bucket-1:myindex:timings/storage_snapshot_create":"195 76812580432 3211135522311963796","bucket-1:myindex:total_scan_duration":0,"bucket-1:num_mutations_queued":36775158,"bucket-1:num_nonalign_ts":148,"bucket-1:ts_queue_size":0,"memory_quota":31457280000,"memory_used":18572451840,"needs_restart":false,"num_connections":0,"timings/stats_response":"8655 1433404644 266140963546380"}